Qwen3.8-27B UD-Q4_K_XL on one RTX 5090: llama.cpp b12359 with Unsloth's MTP head

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-13. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 4 slots (the llama-server default, -np auto). C8 exceeds the slots: the extra requests wait in the queue, and they were measured anyway. What ran. llama.cpp in the prebuilt Linux package b12359 (CUDA 13.5, commit b1b770d90) of ggml-org. This is the build that the v1.0.0 release names: the release page has no binaries, only a file that names b12359; the v1.0.0 tag points to the same commit, b1b770d90. It served Qwen3.8-27B in Unsloth's GGUF file UD-Q4_K_XL (Dynamic v3.1, revision 4af51b35). The speculation is the MTP head of the same Unsloth repository (MTP/mtp-Qwen3.8-27B-Q4_0.gguf), started with --spec-type draft-mtp --spec-draft-n-max 2, as Unsloth's MTP guide says. Where this run differs from the author's recipe. (1) The speed test ran with the server defaults for window and slots (4 slots share one KV pool; 31,216 MiB of GPU memory in use after loading, projector included). The test levels with 8 requests at once exceed the 4 slots, so the extra requests wait in the queue. (2) The quality tests ran with -np 1, because llama-server b12359 with draft-mtp and 4 slots stopped in the first MMLU-Pro run (message: common.cpp:1634: failed to remove sequence 0 with p0=272, p1=-1, 2 of 2 attempts). With -np 1 it ran through. The numbers of that aborted attempt are not shown. (3) The model, the projector and the MTP head come from local files instead of -hf, and --cache-ram 0, --no-webui and --metrics are ours. Against the authors' numbers. We found no decode speed published by ggml-org or Unsloth for this model on an RTX 5090 (checked on 13 November 2026), so there is no figure of the authors next to ours. The context steps at 16K, 32K, 64K and 128K all fit the window.

python3 speed2.py --base-url http://127.0.0.1:36467/v1 --label llama10-5090-mtp-1311-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/llama10-5090-mtp-1311-speed-r0/llama10-5090-mtp-1311-speed-r0/speed2 --contract $RUN/llama10-5090-mtp-1311-speed-r0/llama10-5090-mtp-1311-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source unsloth/Qwen3.8-27B-GGUF README.md:83-84 @4af51b35 (Thinking 1.0/0.95/20/0.0/presence 0.0; Instruct 0.7/0.8/20/0.0/presence 1.5, repetition_penalty 1.0), same as Qwen/Qwen3.8-27B; our parts: thinking off = Instruct profile --model qwen38-27b-llamacpp --no-cache-prompt --server-log $HOME/llama10-run/logs/server.log --engine-version-file $RUN/engine-version.txt --server-slots 4 --server-slots-source llama-server default of b12359 (-np auto, tools_server_README.md:178); the log of the probe start Nov 13, 2026 shows 'n_slots = 4' (kv_unified, one shared KV pool) --server-ctx 153856 --server-ctx-source Author default (llama-server b12359: -c 0 = model value, -fit on adjusts to device memory, tools_server_README.md:50,89): server log of the probe start Nov 13, 2026 'n_slots = 4, n_ctx_slot = 153856, kv_unified = true' (Q4_K_XL + mmproj F16 + MTP head on the RTX 5090, 31.2 GB used). Nothing set by us.

How well it answers

The quality tests ran with -np 1, not with the 4 slots of the speed test (reason under the speed results: the 4-slot server aborted in MMLU-Pro). The server log of that run says n_slots = 1 and n_ctx_slot = 173,824. Thinking off (enable_thinking false in every request), the instruct sampling of the model card (temperature 0.7, top-p 0.8, top-k 20, min-p 0, presence penalty 1.5), seed 42, 1 request at a time. Answer limits: 48 tokens for MMLU-Pro and GPQA (one letter), 4,096 for IFEval, 128 for the long-text tasks (RULER). A cut-off answer counts as wrong: 1 answer of the long-text test at 32K was cut off. Raw numbers only, no overall score. GPQA ran on the public mirror fingertap/GPQA-Diamond (198 questions), because the original needs a login.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-13.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-mtp-llamacpp-b12359-5090-1311.json --parts mmlupro --job llama10-5090-mtp-np1-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-mtp-llamacpp-b12359-5090-1311.json --parts gpqa --job llama10-5090-mtp-np1-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-mtp-llamacpp-b12359-5090-1311.json --parts ifeval --job llama10-5090-mtp-np1-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-mtp-llamacpp-b12359-5090-1311.json --parts ruler --job llama10-5090-mtp-np1-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-mtp-llamacpp-b12359-5090-1311.json --parts ruler --job llama10-5090-mtp-np1-1311

Also made this possible

• Model files on Hugging FaceUnsloth AI@unslothai Hugging Face GitHub Website
• Engine (llama.cpp, prebuilt package b12359)ggml-org (llama.cpp)@ggml_org GitHub Hugging Face Website
• Model (Qwen3.8-27B)Qwen@Alibaba_Qwen Hugging Face GitHub Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5090 32 GB (one machine), driver 610.43.02, kernel 6.12.95 (Debian 13)

Model

Qwen3.8-27B UD-Q4_K_XL GGUF (unsloth/Qwen3.8-27B-GGUF @4af51b35, Dynamic v3.1) + MTP head Q4_0 (same repo)

Model files on Hugging Face

huggingface.co/unsloth/Qwen3.8-27B-GGUF

Model (Qwen3.8-27B)

huggingface.co/Qwen/Qwen3.8-27B

Engine

ggml-org's llama.cpp package b12359 with Unsloth's GGUF and its MTP head (draft-mtp) — the program that loads the model and answers requests · github.com/ggml-org/llama.cpp/releases/tag/b12359

Recipe (MTP guide: --spec-type draft-mtp --spec-draft-n-max 2) and model files (UD-Q4_K_XL, MTP head, Dynamic v3.1, revision 4af51b35)

unsloth.ai/docs/models/mtp by Unsloth AI

Steps

1. Get the recipe, the engine and the model files from the list above. The command also uses the file mmproj-F16.gguf (the vision part of the model); the list has no separate link for it.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   llama-server \
     -m Qwen3.8-27B-UD-Q4_K_XL.gguf \
     --mmproj mmproj-F16.gguf \
     --spec-type draft-mtp \
     -md mtp-Qwen3.8-27B-Q4_0.gguf \
     --spec-draft-n-max 2 \
     --alias qwen38-27b-llamacpp \
     --host 127.0.0.1 \
     --port 8888 \
     --cache-ram 0 \
     --no-webui \
     --metrics

   What the parts mean

   • -m Qwen3.8-27B-UD-Q4_K_XL.ggufthe model file it loads
   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --cache-ram 0the most system RAM, in MiB, used to keep earlier prompts for reuse (0 turns this off)
   • --no-webuiturns off the built-in chat web page; only the API answers
   • --metricsturns on a /metrics page with server statistics (Prometheus format)
   • --mmproj mmproj-F16.ggufthe multimodal projector: the extra file that lets the model read images
   • --spec-…settings for speculative decoding: a draft guesses the next tokens and the model checks them (--spec-type: the kind of draft, --spec-draft-n-max: the most tokens drafted per step)
   • --alias qwen38-27b-llamacppthe model name clients use in their requests

llama.cpp b12359 @b1b770d90 (prebuilt package ubuntu-cuda-13.5-x64 of ggml-org, the build that the v1.0.0 release names; the v1.0.0 tag points to the same commit), MTP draft-mtp n-max 2, f16 KV; llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf --mmproj mmproj-F16.gguf --spec-type draft-mtp -md mtp-Qwen3.8-27B-Q4_0.gguf --spec-draft-n-max 2 --alias qwen38-27b-llamacpp --host 127.0.0.1 --port 8888 --cache-ram 0 --no-webui --metrics (speed test; the quality tests ran with the same command plus -np 1. Unsloth's MTP guide: --spec-type draft-mtp --spec-draft-n-max 2, nothing else; the server chooses window and slots itself; local files instead of -hf, so the projector and the MTP head from the same Unsloth repository are named with --mmproj and -md; --cache-ram 0, --no-webui, --metrics and the alias are ours. The commands under the quality cells name the card of the speed run; the quality jobs ran with the card whose name ends in -np1, which differs only by -np 1)