Qwen3.8-27B UD-Q4_K_XL on one RTX 5090: llama.cpp b12359 with a DFlash2 draft

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-13. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 1 slot. C2, C4 and C8 exceed it: the extra requests wait in the queue, and they were measured anyway. What ran. llama.cpp in the prebuilt Linux package b12359 (CUDA 13.5, commit b1b770d90) of ggml-org. This is the build that the v1.0.0 release names: the release page has no binaries, only a file that names b12359; the v1.0.0 tag points to the same commit, b1b770d90. It served Qwen3.8-27B in Unsloth's GGUF file UD-Q4_K_XL (Dynamic v3.1, revision 4af51b35). The draft is the DFlash2 file dflash-Qwen3.8-27B-Q4_0.gguf from ggml-org/Qwen3.8-27B-GGUF (revision c1869413), which the model card of that repository lists as converted from incoai/Qwen3.8-27B-DFlash2; the server starts it with --spec-type draft-dflash. Where this run differs from the author's example. (1) --fit off, -np 1 and -c 131072 are ours, and the vision projector is not loaded (--no-mmproj). Reason: with the server's default memory fit (-fit on), the start with the DFlash draft ran out of GPU memory in our start attempts (cudaMalloc failed on a buffer of 1,033 MiB), also with the projector left out and with -np 1. With --fit off and the default number of slots, the cache for the recurrent state alone asked for 9,576 MiB and failed. With --fit off, -np 1 and -c 131072 the server loaded (29,082 MiB of GPU memory in use after loading). The window of 131,072 tokens is our choice: it is the largest step of our speed test, so every context step fits. The 128K step ran with a prompt of 130,048 tokens (the rule of our speed test: window minus 1,024). (2) The model and the draft come from local files instead of -hf. --cache-ram 0 (RAM prompt cache off), --no-webui and --metrics are ours. Against the authors' numbers. We found no decode speed published by ggml-org or Unsloth for this model on an RTX 5090 (checked on 13 November 2026), so there is no figure of the authors next to ours.

python3 speed2.py --base-url http://127.0.0.1:40759/v1 --label llama10-5090-dflash-1311-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/llama10-5090-dflash-1311-speed-r0/llama10-5090-dflash-1311-speed-r0/speed2 --contract $RUN/llama10-5090-dflash-1311-speed-r0/llama10-5090-dflash-1311-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source unsloth/Qwen3.8-27B-GGUF README.md:83-84 @4af51b35 (Thinking 1.0/0.95/20/0.0/presence 0.0; Instruct 0.7/0.8/20/0.0/presence 1.5, repetition_penalty 1.0), same as Qwen/Qwen3.8-27B; our parts: thinking off = Instruct profile --model qwen38-27b-llamacpp --no-cache-prompt --server-log $HOME/llama10-run/logs/server.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source FORCED, named deviation from -np auto: with --fit off and 4 slots the rs cache alone asks 9.5 GiB (OOM); -np 1 loads. Probe Nov 13, 2026 22:43: 'n_slots = 1, n_ctx_slot = 131072'. --server-ctx 131072 --server-ctx-source FORCED, named deviation: with -fit on the DFlash draft does not start in b12359 (log: 'dflash requires ctx_other to be set', then cudaMalloc OOM); with --fit off -np 1 -c 131072 it loads (probe Nov 13, 2026 22:43, 29.1 GB used). Window = the model-card-independent 128K step (our choice, same step as the TensorFold 131072 card).

How well it answers

Thinking off (enable_thinking false in every request), the instruct sampling of the model card (temperature 0.7, top-p 0.8, top-k 20, min-p 0, presence penalty 1.5), seed 42, 1 request at a time, the same server start as the speed test (1 slot, DFlash2 draft). Answer limits: 48 tokens for MMLU-Pro and GPQA (one letter), 4,096 for IFEval, 128 for the long-text tasks (RULER). A cut-off answer counts as wrong: 1 answer of the long-text test at 32K was cut off. Raw numbers only, no overall score. GPQA ran on the public mirror fingertap/GPQA-Diamond (198 questions), because the original needs a login.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-13.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-dflash2-llamacpp-b12359-5090-1311.json --parts mmlupro --job llama10-5090-dflash-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-dflash2-llamacpp-b12359-5090-1311.json --parts gpqa --job llama10-5090-dflash-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-dflash2-llamacpp-b12359-5090-1311.json --parts ifeval --job llama10-5090-dflash-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-dflash2-llamacpp-b12359-5090-1311.json --parts ruler --job llama10-5090-dflash-1311

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-ud-q4kxl-dflash2-llamacpp-b12359-5090-1311.json --parts ruler --job llama10-5090-dflash-1311

Also made this possible

• Engine (llama.cpp, prebuilt package b12359)ggml-org (llama.cpp)@ggml_org GitHub Hugging Face Website
• DFlash2 draft file (dflash-Qwen3.8-27B-Q4_0.gguf, revision c1869413)ggml-org (llama.cpp)@ggml_org Hugging Face GitHub Website
• Draft model that the file is converted from (DFlash2, named on the ggml-org model card)Inco AIHugging Face GitHub Website
• Model files (Qwen3.8-27B-UD-Q4_K_XL.gguf, Dynamic v3.1, revision 4af51b35)Unsloth AI@unslothai Hugging Face GitHub Website
• Model (Qwen3.8-27B)Qwen@Alibaba_Qwen Hugging Face GitHub Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5090 32 GB (one machine), driver 610.43.02, kernel 6.12.95 (Debian 13)

Model

Qwen3.8-27B UD-Q4_K_XL GGUF (unsloth/Qwen3.8-27B-GGUF @4af51b35, Dynamic v3.1) + DFlash2 drafter Q4_0 (ggml-org/Qwen3.8-27B-GGUF @c1869413, from incoai/Qwen3.8-27B-DFlash2)

DFlash2 draft file (dflash-Qwen3.8-27B-Q4_0.gguf, revision c1869413)

huggingface.co/ggml-org/Qwen3.8-27B-GGUF

Draft model that the file is converted from (DFlash2, named on the ggml-org model card)

huggingface.co/incoai/Qwen3.8-27B-DFlash2

Model files (Qwen3.8-27B-UD-Q4_K_XL.gguf, Dynamic v3.1, revision 4af51b35)

huggingface.co/unsloth/Qwen3.8-27B-GGUF

Model (Qwen3.8-27B)

huggingface.co/Qwen/Qwen3.8-27B

Engine

ggml-org's llama.cpp package b12359 with Unsloth's GGUF and the DFlash2 sidecar file of ggml-org — the program that loads the model and answers requests · github.com/ggml-org/llama.cpp/releases/tag/b12359

Recipe (the DFlash example of docs/speculative.md)

github.com/ggml-org/llama.cpp/blob/b12359/docs/speculative.md by ggml-org (llama.cpp)

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   llama-server \
     -m Qwen3.8-27B-UD-Q4_K_XL.gguf \
     --no-mmproj \
     --spec-type draft-dflash \
     -md dflash-Qwen3.8-27B-Q4_0.gguf \
     --spec-draft-n-max 15 \
     -np 1 \
     -c 131072 \
     -fa on \
     --jinja \
     --fit off \
     --alias qwen38-27b-llamacpp \
     --host 127.0.0.1 \
     --port 8888 \
     --cache-ram 0 \
     --no-webui \
     --metrics

   What the parts mean

   • -m Qwen3.8-27B-UD-Q4_K_XL.ggufthe model file it loads
   • -c 131072the context window in tokens: how much text the model can see at once
   • -np 1how many requests it answers at the same time
   • --jinjauses the chat template that comes with the model
   • -fa onturns on FlashAttention, a faster way to compute attention
   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --cache-ram 0the most system RAM, in MiB, used to keep earlier prompts for reuse (0 turns this off)
   • --no-webuiturns off the built-in chat web page; only the API answers
   • --metricsturns on a /metrics page with server statistics (Prometheus format)
   • --spec-…settings for speculative decoding: a draft guesses the next tokens and the model checks them (--spec-type: the kind of draft, --spec-draft-n-max: the most tokens drafted per step)
   • --alias qwen38-27b-llamacppthe model name clients use in their requests

llama.cpp b12359 @b1b770d90 (prebuilt package ubuntu-cuda-13.5-x64 of ggml-org, the build that the v1.0.0 release names; the v1.0.0 tag points to the same commit), draft-dflash n-max 15 (clamped), f16 KV; llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf --no-mmproj --spec-type draft-dflash -md dflash-Qwen3.8-27B-Q4_0.gguf --spec-draft-n-max 15 -np 1 -c 131072 -fa on --jinja --fit off --alias qwen38-27b-llamacpp --host 127.0.0.1 --port 8888 --cache-ram 0 --no-webui --metrics (the DFlash example of docs/speculative.md of llama.cpp b12359, which reads: llama-server -m <model>.gguf -md <draft>.gguf --spec-type draft-dflash --spec-draft-n-max 15 -fa on --jinja; plus our --fit off, -np 1, -c 131072 and --no-mmproj, with the reason under the speed results; local files instead of -hf; --cache-ram 0, --no-webui, --metrics and the alias are ours)