tqLLM 1.5.0 by turbo-tan: Qwen3.8-27B TQ_XS with DFlash2 drafts and a tq3 KV cache on a 16 GB card

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-11. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 1 slot (from the author's recipe). C2, C4, C8 exceed the slots: the extra requests wait in the queue, and they were measured anyway. The author's recipe. The model card of the weights (Hugging Face YTan2000/Qwen3.8-27B-TQ, revision 3eef6f50; the line numbers are those of its README.md) says that TQ_XS and TQ_S target 16 GB GPUs (lines 23 and 24) and lists Qwen3.8-27B-TQ_XS.gguf as the smallest file, 11.85 GB, for 16 GB GPUs with the most room for context (line 43). Its run example (lines 66 to 73) is a docker run with MODEL_FILE, DRAFT_MODEL_FILE, DRAFT_VOCAB_FILE, KV_FORMAT=tq3, CONTEXT=32768, REASONING=on and MAX_TOKENS=4096 (the example shows the file TQ_S and the image ytan2000/tqllm:1.0.0; line 58 says to replace the file name for another size). The README of tqLLM (turbo-tan/tqllm at the commit of release 1.5.0, line 96) says that a version tag such as 1.5.0 pins the image. I ran the 1.5.0 image with the TQ_XS file, the drafter and the token list of the card.

Release and files. The engine is the author's official Docker image, not a build of my own: tag 1.5.0 is the index digest sha256:b9126efd065467361974f8e4411c34f1cee4c2b2bedc42c44fa25376682e1516 (image created 10 Nov 2026, 12:23 UTC; the registry still resolved the tag to this digest on 11 Nov 2026), and the GitHub release v1.5.0 of turbo-tan/tqllm was published on 10 Nov 2026, 12:39 UTC by turbo-tan (it has no files attached; the image is its package). The server reports engine commit 6314171, which is the commit of the tag v1.5.0. The model file is Qwen3.8-27B-TQ_XS.gguf, 11,853,623,104 bytes, sha256 e7e691c07a67e9986c1247cd3cf55ba7d5c12366b73bd915a8d3981a9c6f798a, and the drafter file is drafters/Qwen3.8-27B-DFlash2-TQ3.gguf, 785,487,488 bytes, sha256 7f12a657b8c039f6748eb665c650fd0de47284cd8975847fa4e6757cf77a07bd; both hashes are the values Hugging Face lists for the files, and the token list is mtp-vocab/atx_65536.txt (Hugging Face lists no hash for it). The release notes and the README of tqLLM 1.5.0 (README line 22) call the file 'Qwen3.8-27B TQ3_XS', while Hugging Face holds only TQ_XS (the same size, 11.85 GB). I ran the Hugging Face file; that the two names are one file is not proven byte by byte. The drafter's own notice (drafters/NOTICE, lines 3 and 4) says that it is a TQ3 conversion, made by the tqLLM authors, of z-lab/Qwen3.8-27B-DFlash2. The credit table of the model card (lines 30 to 35) also names unsloth (the BF16 source GGUF), froggeric (the chat template) and Google Research (TurboQuant, the lineage of the TQ3 formats).

Differences from the author's recipe, all named: (1) the context is 132,096 tokens where the card's example has 32,768 (my choice, so that the 128K step of the speed test fits; the image default is 16,384); (2) MAX_TOKENS is 32768 where the example has 4096 (the quality tests need answers up to 32,768 tokens); (3) the file is TQ_XS where the example shows TQ_S, and the image is 1.5.0 where the example shows 1.0.0; (4) how I start the container: only the 5060 Ti is visible to it (selected by its UUID), it runs on the host network with --host 127.0.0.1 --port 8888 instead of the image's port 8086, it is limited to 12 GB of RAM without swap, and a watchdog of my start script would stop it if the host had less than 4,000 MiB of RAM available (the watchdog's log is not among the published files); (5) thinking is switched per request (the server runs with REASONING=on as in the example; the speed test and the four quality tests with thinking off send enable_thinking false); (6) the author names no sampling, and the author's own benchmark numbers are greedy (docs/BENCHMARKS.md, lines 10 and 33), while mine use the sampling of the Qwen3.8-27B model card, so my method differs from the author's. The command shown above is the one my start script ran; it also holds the author's variables KV_FORMAT=tq3, the drafter and the token list, and the image default of 1 slot (PARALLEL is not set).

The headline 79.5 tok/s is the mean of the code prompt (84.2) and the prose prompt (74.8), medians of two runs each (spread (CV) of 5.0 % for the code runs and 6.7 % for the prose runs, as in the table). The drafter accepted 26.8 % of the tokens it proposed in the four requests of the decode test (1,341 of 4,998, the server's own counters). The long-answer test (4,459 tokens) ended by itself (finish reason stop). Requests at once: the server has 1 slot (the image default), so 2, 4 and 8 requests wait for it and queue (the tool measured those levels anyway). The total is 60.7, 61.7, 64.4 and 63.1 tok/s for 1, 2, 4 and 8 requests at once; the speed of one request, counted from sending to the last token and so including the wait, is 60.7, 45.5, 26.3 and 14.2 tok/s. The 1-request figure includes the prompt time, so it is below the decode speed. These are measured rates for this setup, not a statement about how it would scale on other hardware.

Cache check: the tool's check could not confirm that no prompt tokens were served from a cache, because the server's replies do not report cached tokens (unknown in all 61 requests it examined). Every request sent cache_prompt false, and I did not check the cache behaviour in another way.

Long prompts: all four steps (16K, 32K, 64K and 128K tokens) ran at the server context of 132,096 tokens, and the 128K step is a full 131,072 tokens: its first token came after 180.7 s (prefill 725.3 tok/s), and decode at that depth is 45.2 tok/s (code) and 40.0 tok/s (prose), against 76.3 and 74.5 tok/s at 32K. Every step has two valid runs per prompt; the largest spread between the two runs of a step is 11.2 % (CV, prose prompt at 32K). The headline prefill figures are for 2K, 8K and 32K prompts.

The author's own numbers for this card (README of tqLLM 1.5.0, line 22: an RTX 5060 Ti 16 GB on native Windows with the file named TQ3_XS, prompts about 1,200 tok/s and answers 160 tok/s with the DFlash2 drafter; model card, lines 109 to 117: RTX 5060 Ti 16 GB under Windows 11 / WSL 2, TQ_XS with the DFlash2-TQ3 drafter, 32K context, thinking on, median of 5 runs, 175.9 tok/s for the reasoning prompt and 98.8 tok/s for the code prompt) come from a different method than mine (their operating system, their prompts, thinking on and, for the benchmark page, greedy decoding; mine: Linux, thinking off, the sampling of the model card, my fixed prompts). I did not run the author's method, so my numbers neither confirm nor contradict theirs.

The first try of this setup (11 Nov 2026, ended 06:13, job tqllm15-16gb-1111) failed in every part (the speed test, the four quality tests, RULER and GPQA 30 are listed as failed; the memory test did not run) and has no register rows; its raw files are not published. The speed run shown is the later one (11 Nov 2026, 07:16 to 07:50). Its guard (a sample every 0.5 s; a step is repeated after 2 samples in a row above 25 % of one core; the box's own background services and the engine process are on its ignore list) repeated 2 of its 38 steps, each once: step 'C4, round 1' (a git job on the box's own files, up to 92.0 % of one core) and step '128K, prose, run 1' (the same kind of job, up to 94.2 %); the guard took 4,099 samples. The highest reading of any step was 138.0 % of one core (a git job on the box's own files, in the first prose run of the 64K step); that step was not repeated. The tool's separate quiet check (one sample every 10 seconds; it lists the engine process too) listed a busiest process above 25 % of one core in all 204 samples: in 195 of them it was the engine process of this setup itself (up to 94.9 %), and in the other 9 samples another program (up to 137.0 %; 5 the box's own indexing service, 2 an index rebuild of the box's own services, 1 a git job and 1 a process-listing helper). Nothing was removed. The GPU reading of that quiet check is not that of the measured card: it stays at one constant value of 5,602 MiB, which is the memory use of another GPU in the machine, so there is no memory reading of the measured card during the speed test.

python3 speed2.py --base-url http://127.0.0.1:42763/v1 --label tqllm15-16gb-1111c-speed-r0 --prompts $BENCH/repos/bench-v2/prompts-v2.json --output $RUN/tqllm15-16gb-1111c-speed-r0/tqllm15-16gb-1111c-speed-r0/speed2 --contract $RUN/tqllm15-16gb-1111c-speed-r0/tqllm15-16gb-1111c-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/Qwen3.8-27B-EXL3-2.5bpw/tokenizer.json --tokenizer-python $BENCH/venv/bin/python --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source The author names no default sampling (neither tqllm.com/docs.html nor the model card; the author's own benchmark numbers are greedy, docs/BENCHMARKS.md lines 10 and 33). Sampling of the Qwen3.8-27B model card, as for the other Qwen3.8 cards. TO VERIFY: whether tqserve reads min_p and presence_penalty (tqllm.com/docs.html: 'sampling uses at most 16 candidates'). --model qwen3.8-27b-tq-xs --no-cache-prompt --server-log $HOME/tqllm/logs/server.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source docker/entrypoint.sh @6314171f: --np ${PARALLEL:-1} (default 1); the model card and the start page do not set PARALLEL (their example calls have no PARALLEL). --server-ctx 132096 --server-ctx-source The authors' sources disagree: the Docker entrypoint (docker/entrypoint.sh @6314171f) has CONTEXT 16384 as its default, the model card example has CONTEXT=32768, and tqllm.com/performance.html (command line, RTX 5060 Ti) says 'Default context is 204,800' with the memory fit and long-context rows up to 248K. The v2 tests need more than 16384/32768 (a LiveCodeBench answer limit of 32768, long prompts up to 128K). Chosen: 132096 = the smallest context that holds the 128K step (131,072 + 512 answer tokens); the engine places the KV by its own memory fit (its start-up log of the run: the oldest 11,264 positions in host memory, 0.13 GB; at 204800 it is 73,728). Forced deviation from the author's default, named on this page.

How well it answers

RULER 8K/32K is not reported for this setup: with the sampling of the model card (temperature above 0) and the DFlash2 drafter on, the 8K and 32K prompts of the test were answered with an empty reply of 1 token in all but one case, one 8K prompt sent afterwards at temperature 0 was answered correctly, and the sampling was not changed for the scored run. Five tests ran: four with thinking off and one (GPQA 30) with thinking on, and answers cut off at the token limit count as wrong (4 of 40 in LiveCodeBench, 10 of 30 in GPQA 30). The notes below say how the numbers were taken.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-11.

Note from the run: MMLU-Pro (400 questions), GPQA Diamond (198), IFEval (200) and LiveCodeBench (40 tasks) ran as one run, together with the RULER part that is not reported (see above) (11 Nov 2026, 08:18 to 09:58), with thinking off and the instruct sampling of the Qwen3.8-27B model card (temperature 0.7, top-p 0.8, top-k 20, min-p 0, presence penalty 1.5), seed 42, one question at a time (the server has one slot). The answer limits are 48 tokens for MMLU-Pro and GPQA Diamond (the tool's default for a one-letter answer; the median answer was 2 tokens in both tests and none was cut off), 4,096 for IFEval and 32,768 for LiveCodeBench. GPQA 30 ran with thinking on and the thinking sampling of the card (temperature 1.0, top-p 0.95, top-k 20, min-p 0), an answer limit of 16,384 tokens and one question at a time, as a run of its own (11 Nov 2026, 09:58 to 10:52). GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 rows), because the original is gated and no Hugging Face token was available.

Thinking: the model also runs with thinking on, and the maker of the base model gives a separate sampling profile for it. The speed test and four of the five quality tests run with thinking off, so they may understate the model with thinking on; I have not measured by how much. Only GPQA 30 ran with thinking on: 18 of 30. The median answer of that test used 7,612 tokens and 88.6 seconds per question (one question at a time), and 10 of the 30 answers were cut off at the limit of 16,384 tokens.

Cut-off answers count as wrong: 0 in MMLU-Pro, 0 in GPQA Diamond, 0 in IFEval, 4 of the 40 in LiveCodeBench (limit 32,768 tokens; the median answer used 4,000 tokens; the tool counts these as having no code) and 10 of the 30 in GPQA 30 (limit 16,384 tokens).

Earlier attempts: besides the first try of that morning (see the speed notes) the tool refused three quality starts of this job (11 Nov 2026, 07:50, 07:53 and 08:16) at its quiet check, because it listed the engine's own container process as a foreign process using the card; nothing was sent to the server in those starts, and they have no register rows. Afterwards I told the tool that the container is the server under test (a setting of my card, not of the engine), and the runs shown started at 08:18 and 09:58. One engine process served both runs (engine-start.log in the raw folder of the first run: the context is 132,096 tokens and the memory fit is that of the speed test).

The tool's contract file and log of the first run (both linked in the table) still list the part that is not reported with their own entries; the numbers of the table are those of the summary files.

Foreign load: the quality tests have no guard; their sampling file lists the busiest other process every 10 seconds, and the engine process does not appear in these two files (0 of 920 samples). First run (MMLU-Pro, GPQA Diamond, IFEval, LiveCodeBench and the RULER part): 24 of 601 samples above 25 % of one core (up to 277.0 %; 12 the box's own indexing service, 3 an index rebuild of the box's own services, 3 a git job on the box's own files, 3 the sandbox of the code test itself, 1 a container start, 1 a process-listing helper and 1 a disk-usage listing). The three highest readings were a git job on the box's own files: 277.0 % at 08:50 and 243.0 % at 09:25 (both in the LiveCodeBench part) and 203.0 % at 09:35 (in the RULER part). GPQA 30 with thinking: 11 of 319 samples (up to 105.0 %; 7 the indexing service, 2 a file search, 1 a git maintenance job and 1 a process-listing helper), the highest being the indexing service at 10:09. The memory in use on the measured card stayed between 14,762 and 14,852 MiB in both runs. Nothing was repeated or removed. The tool's contract files of both runs name the busiest process in the same way (their remark 'foreign CPU load during the run').

bash tools/quality2/measure_all.sh tools/quality2/cards/tqllm150-qwen38-27b-tq-xs-dflash2-tq3kv-5060ti.json --parts mmlupro --job tqllm15-16gb-1111c

bash tools/quality2/measure_all.sh tools/quality2/cards/tqllm150-qwen38-27b-tq-xs-dflash2-tq3kv-5060ti.json --parts gpqa --job tqllm15-16gb-1111c

bash tools/quality2/measure_all.sh tools/quality2/cards/tqllm150-qwen38-27b-tq-xs-dflash2-tq3kv-5060ti.json --parts ifeval --job tqllm15-16gb-1111c

bash tools/quality2/measure_all.sh tools/quality2/cards/tqllm150-qwen38-27b-tq-xs-dflash2-tq3kv-5060ti.json --parts lcb --job tqllm15-16gb-1111c

bash tools/quality2/measure_all.sh tools/quality2/cards/tqllm150-qwen38-27b-tq-xs-dflash2-tq3kv-5060ti.json --parts think --job tqllm15-16gb-1111c

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. Measured 2026-11-11. Raw files: result of every step · log · contract.

Note from the run: The server was restarted for every size with CONTEXT set to that size (1 slot, KV tq3, the same start script as in the speed test), and one prompt that fills about 95 % of the size was sent (prompts of 3,798 to 195,524 tokens, filling 92.7 % to 96.4 % of the size; at most 96 answer tokens); the model had to answer. All seven sizes, 4,096 to 204,800 tokens, loaded and answered (every answer ended by itself), so 204,800 tokens is the largest size that answered; it is the highest size tested, not a limit of the setup. The memory number is the GPU memory in use on the whole card (the peak of the test): it is 14.39 GiB at 204,800 tokens, the lowest of the seven sizes, and between 14.39 and 15.37 GiB over all sizes (highest at 32,768 tokens). The engine's own memory fit (engine-start-c<size>.log in the raw folder of this test) kept no KV positions in host memory at 65,536 tokens and below, the oldest 54,272 of 131,072 positions at 131,072 tokens (0.67 GB in host memory, 1.29 GB on the card) and the oldest 73,728 of 204,800 positions at 204,800 tokens (1.01 GB in host memory, 2.07 GB on the card). The decode speed shown (13.8 tok/s at 204,800) is that of one short reply of 86 tokens after a prompt of 195,524 tokens; it is not comparable with the decode speeds of the speed test. The size of 131,072 tokens shows how different the two tests are: the memory test measured 15.2 tok/s there (a reply of 90 tokens after a prompt of 125,892 tokens, server context 131,072), while the speed test measured 45.2 tok/s (code prompt) and 40.0 tok/s (prose prompt) after a prompt of exactly 131,072 tokens with 512 answer tokens at a server context of 132,096. Server context, prompt and answer length differ, and I did not investigate the difference. The prefill column is the server's own report for that one request.

bash tools/quality2/measure_all.sh tools/quality2/cards/tqllm150-qwen38-27b-tq-xs-dflash2-tq3kv-5060ti.json --parts fit --job tqllm15-16gb-1111c

How it plays the Gauntlet

One solo race, season seed 21, one attempt, no help.

Game rank as of Nov 11, 2026. The rank is the order of the Gauntlet finish times in this memory class. Race page and raw files.

The race ran the recipe named on its race page, with the sampling and answer limit of its lane config. The engine build of the race is not recorded in the race files.

Result (result.json) Race log (log.jsonl) Model answers (io.jsonl) System prompt Exact command Verify output Model config Race page

Also made this possible

• Base model Qwen3.8-27B (revision 37561816)Qwen@Alibaba_Qwen Hugging Face GitHub Website
• Draft model (DFlash2; the source of the Qwen3.8-27B drafter is incoai/Qwen3.8-27B-DFlash2)Inco AI@inco_ai Hugging Face GitHub Website
• Speculative decoding method (DFlash) and the Hugging Face mirror of the drafter (z-lab/Qwen3.8-27B-DFlash2)z-lab (Zhijian Liu)@zhijianliu_ Hugging Face GitHub
• TQ3 conversion of the drafter (the file Qwen3.8-27B-DFlash2-TQ3.gguf is the TQ3 conversion of z-lab/Qwen3.8-27B-DFlash2 made by the tqLLM authors; drafters/NOTICE, lines 3 and 4)turbo-tan (tqLLM authors)GitHub Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5060 Ti 16 GB (one GPU), driver 595.71.05, Docker. The container was limited to 12 GB of RAM without swap (--memory 12g --memory-swap 12g). At the 132,096-token context the engine's own memory fit (its start-up log, engine-start.log in the raw folder of the run) moved the int8 copy of the output layer (1.43 GB) and the token embeddings (0.72 GB) into host memory and kept the oldest 11,264 of the 132,096 KV positions per sequence there too (0.13 GB in host memory, 1.83 GB on the card)

Model

Qwen3.8-27B TQ_XS (YTan2000/Qwen3.8-27B-TQ @3eef6f50, 11.85 GB) + DFlash2-TQ3 drafter (0.79 GB)

Base model Qwen3.8-27B (revision 37561816)

huggingface.co/Qwen/Qwen3.8-27B

DFlash2 draft model and method (the file Qwen3.8-27B-DFlash2-TQ3.gguf is the TQ3 conversion of z-lab/Qwen3.8-27B-DFlash2 made by the tqLLM authors)

huggingface.co/z-lab/Qwen3.8-27B-DFlash2

Engine

tqLLM 1.5.0 (the author's official Docker image), the TQ_XS file of YTan2000/Qwen3.8-27B-TQ (revision 3eef6f50), DFlash2-TQ3 drafter, one RTX 5060 Ti 16 GB, 132,096 tokens of context, 1 slot, text only — the program that loads the model and answers requests · github.com/turbo-tan/tqllm/releases/tag/v1.5.0Build and settings we used: tqLLM 1.5.0 (official Docker image ghcr.io/turbo-tan/tqllm:1.5.0, engine commit 6314171), DFlash2-TQ3 drafter, KV tq3, 1 slot

Steps

1. Get the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   docker run \
     --rm \
     --name tqllm \
     --gpus "device=<GPU-UUID>" \
     --network host \
     --memory 12g \
     --memory-swap 12g \
     -v <models folder>:/models:ro \
     -e \
     MODEL_FILE=/models/Qwen3.8-27B-TQ_XS.gguf \
     -e \
     DRAFT_MODEL_FILE=/models/drafters/Qwen3.8-27B-DFlash2-TQ3.gguf \
     -e \
     DRAFT_VOCAB_FILE=/models/mtp-vocab/atx_65536.txt \
     -e \
     KV_FORMAT=tq3 \
     -e \
     CONTEXT=132096 \
     -e \
     REASONING=on \
     -e \
     MAX_TOKENS=32768 ghcr.io/turbo-tan/tqllm:1.5.0 \
     --host 127.0.0.1 \
     --port 8888

   What the parts mean

   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --name tqllmthe name of the container

tqLLM 1.5.0 (official Docker image ghcr.io/turbo-tan/tqllm:1.5.0, engine commit 6314171), DFlash2-TQ3 drafter, KV tq3, 1 slot; docker run --rm --name tqllm --gpus "device=<GPU-UUID>" --network host --memory 12g --memory-swap 12g -v <models folder>:/models:ro -e MODEL_FILE=/models/Qwen3.8-27B-TQ_XS.gguf -e DRAFT_MODEL_FILE=/models/drafters/Qwen3.8-27B-DFlash2-TQ3.gguf -e DRAFT_VOCAB_FILE=/models/mtp-vocab/atx_65536.txt -e KV_FORMAT=tq3 -e CONTEXT=132096 -e REASONING=on -e MAX_TOKENS=32768 ghcr.io/turbo-tan/tqllm:1.5.0 --host 127.0.0.1 --port 8888