Mia's one-click kit: Qwen3.8-27B, EXL3 2.5 bpw + MTP

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-11. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 1 slot (from the author's recipe). C2, C4, C8 exceed the slots: the extra requests wait in the queue, and they were measured anyway. Recipe and what differs from it. The setup is Mia's one-click kit (commit bbb11db, the kit's latest commit on 11 Nov 2026, made on 9 Oct 2026) with its own default profile for a 16 GB card: 2.5 bpw with a context of 176,128 tokens with images (kit README line 283; line 311: GPU memory plan 14.7 GB, 176,128 tokens with images, measured by the author on 6 Sep 2026), KV cache in 4 bits, draft method DRAFT=mtp (the kit's default, README line 777), engine ExLlamaV3 1.6.0 (the version the kit requires, line 630; the installed package on the box is exllamav3 1.6.0 in the build +cu128.torch2.11.0, its folder dates from 19 Oct 2026, 02:15, checked on 11 Nov 2026). A small start script of mine writes the kit's .env with exactly these values (MODEL_DIR, CONTEXT_SIZE=176128, CACHE_QUANT=4, GPU_MEM_GB=14.7, DRAFT=mtp, PROFILE=2.5bpw-176k) and calls the kit's own linux/start.sh. Differences from the author's way of starting: the script passes --no-harness (the kit serves /v1 only, no chat page; README line 169) and --no-pick (an internal flag of start.sh that skips the model menu, start.sh lines 66 and 74); the .env also sets UI=no, HOST=127.0.0.1 (README line 774 names this value for keeping /v1 off the network), SETUP=console, and the model keys that the kit's own profile menu writes (MODEL_DIR, HF_TARGET_REPO=turboderp/Qwen3.8-27B-exl3, HF_REVISION=2.50bpw, MODEL_ID; README lines 276 and 277) plus PROFILE_GPU (the GPU's name). PORT (8888), CPU_CACHE_GB (0), VISION (auto) and IMAGE_MAX_PIXELS (1048576) are the kit's defaults. VISION=auto loads the vision tower, as in the kit's 176k-with-images profile; the tests send text only. The kit's start log of a later start with the same settings (11 Nov 2026) shows the MTP head being loaded and 'Images: ON'; the start logs of the speed and quality runs themselves were overwritten by later starts. Not used: DRAFT=dflash2, which the kit offers for 32 GB cards with a 4.0 bpw quant only (README line 777). The model files are turboderp's Qwen3.8-27B-exl3, branch 2.50bpw, revision 0cd10912 (last change of the branch 14 Aug 2026): two weight files of 8,582,745,470 and 3,714,792,495 bytes; on 11 Nov 2026 the sha256 of both local files was compared with the hashes Hugging Face lists, and they are equal. The prompts of the long-prompt steps were counted with the tokenizer file of turboderp's 4.0 bpw branch, because the kit's server has no /tokenize; that file has the same hash on Hugging Face as the file of the 2.5 bpw branch.

The headline 53.0 tok/s is the mean of the code prompt (52.7) and the prose prompt (53.2), medians of two runs each (spread 4.8 % and 2.4 %, as in the table). The speed test sent the instruct sampling of the Qwen model card (the kit's own sampling defaults are temperature 0.6, top-p 0.95, top-k 20 with thinking on; README line 497) with thinking off in every request; the model thinks by default, so a request without that switch would start with reasoning. The long-answer test ended by itself after 4,238 tokens (finish reason stop).

Draft statistics and prompt cache: the setup runs with DRAFT=mtp, but no draft statistics reached the test tool (its contract lists the acceptance rate and the mean accepted length as not available), so the page shows no acceptance rate and I cannot say how many of the drafted tokens were accepted. Every request sent cache_prompt false, and the prefill prompts carry a random nonce of the tool. The tool's own cache check could not confirm the setting: none of the 61 requests it examined reported a cached-token count above zero, but all 61 lacked the field, because the kit's server does not report it. So the check shows no cache hit and does not prove that there was none.

Requests at once: the kit answers one request at a time and queues the others (README lines 497 and 498), so the slot count is 1 and C2, C4 and C8 are measured queues. The total rate is 43.4, 44.3, 44.8 and 43.7 tok/s for 1, 2, 4 and 8 requests; the rate per request is 43.4, 33.5, 18.8 and 9.8 tok/s. The 1-request figure includes the prompt time, so it is below the decode speed.

Long prompts: all four steps ran (16K, 32K, 64K and 128K tokens, exact lengths from the local tokenizer file; the server window is 176,128 tokens, so 131,072 prompt tokens plus 512 answer tokens fit). The first token at 128K comes after 226.4 seconds (prefill 578.9 tok/s); decode there is 39.7 tok/s for the code prompt and 39.3 tok/s for the prose prompt. The memory test below went further and answered a prompt of 168,347 tokens at the full window of 176,128 tokens.

The machine: the box runs its own background services and was not idle. The quiet check of this run (one sample every 10 seconds, the busiest other process each time) saw 235 samples, 5 of them above 25 % of one core: the box's own indexing service in 3 samples (at most 103.0 %), a file search in 1 sample (at most 100.0 %) and a git job in 1 sample (at most 925.0 %). The test tool's own guard (two samples a second, 38 steps) repeated one step: the first try of the second 64K prose run was thrown away because a process named python3 that I did not identify stayed above 25 % of a core for two samples in a row (up to 102.1 %), and the repeat is the value shown; in that repeat the busiest other process was a git job at 94.0 %. The guard repeats a step only when a process stays above 25 % for two samples in a row, and it does not count a fixed list of the box's own background services (among them its indexing service, vector database, container runtime and remote-login service). In 11 of the 38 steps the highest single reading of another process was above 25 % without that, and these steps were kept: git jobs up to 140.0 % (5 steps, the highest in the first 64K prose run), file searches up to 80.0 % (3 steps, the highest in the second round of 8 requests at once) and a Python process from the test tool's own folder, up to 82.4 % (3 steps, the highest in the second 64K code run). Its command line is cut off in the guard files; the tool counts tokens with a local tokenizer file, and I take this process to be that helper, which the guard counts as another process. Nothing else was repeated or removed. Not published: the quiet-check files (quiet-before.txt, quiet-after.txt and ruhe.csv), the guard's sample file and its error file, and the folder of per-request captures (requests/, 128 files, 2 of them from the thrown-away try).

python3 speed2.py --base-url http://127.0.0.1:35403/v1 --label mia25-16gb-1111-speed-r0 --prompts $BENCH/repos/speed-test-v2/prompts-v2.json --output $RUN/mia25-16gb-1111-speed-r0/mia25-16gb-1111-speed-r0/speed2 --contract $RUN/mia25-16gb-1111-speed-r0/mia25-16gb-1111-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/tp2-bench/models/Qwen3.8-27B-EXL3-4.0bpw/tokenizer.json --tokenizer-python $BENCH/venv/bin/python --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source thinking = Mia kit server defaults: MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install@bbb11db:README.md:497 ("Defaults: temperature 0.6, top-p 0.95, top-k 20, thinking on") and tools/serve_openai.py:961-963 (request values win; min_p and presence_penalty are not read by the kit server, 0.0 = Qwen model card). instruct (speed2 etc., thinking off) = Qwen model card: https://huggingface.co/Qwen/Qwen3.8-27B README.md line 253. --model qwen3.8-27b-exl3-2.5bpw --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source Kit: One request at a time; extras queue (README.md:497-499 @bbb11db) --server-ctx 176128 --server-ctx-source card qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti-g1 field ctx

How well it answers

All seven quality tests ran. They are from an earlier run of this setup, on 19 Oct 2026; speed and memory tests are from 11 Nov 2026. Six quality tests ran with thinking off and one (GPQA 30) with thinking on. The notes below say what was checked to call the two runs the same setup and how the numbers were taken.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-19.

Note from the run: Earlier run, same setup: the seven quality tests below are from 19 Oct 2026, three weeks before the speed and memory tests of 11 Nov 2026. They were not repeated on 11 Nov 2026. Both runs used the same setup, checked in these points: the model files (the same folder, unchanged since 30 Sep 2026 by their file dates; their sha256 was compared with Hugging Face's hashes on 11 Nov 2026 and is equal), the kit commit bbb11db (the kit had no later commit on 11 Nov 2026) and engine ExLlamaV3 1.6.0 (the engine version is recorded as the same text in both runs; the package version was not recorded in either run's raw files, but on 11 Nov 2026 I found the installed package to be exllamav3 1.6.0 with a folder date of 19 Oct 2026, 02:15, which is before the quality runs shown here: they started at 18:11 and 19:55 that day), the start command (the register rows of both runs record the same .env values and the same call; the only change in the start script between its backup copy of 20 Oct 2026 and today's is an optional context setting for the memory test, whose default is the same 176,128 tokens, and one printed line), the context of 176,128 tokens, KV cache in 4 bits and GPU memory plan 14.7 GB, DRAFT=mtp, the sampling of the model card and the kit (identical settings files in both runs), seed 42, the box's driver and kernel. The register rows of the two runs list different file hashes for the measuring tool, because its files were edited after 19 Oct 2026; I compared the scripts that ask the questions and score the answers (MMLU-Pro and GPQA, IFEval, LiveCodeBench, RULER and the shared helpers) of 19 Oct with today's: they differ by an option to ask several questions at once (this kit takes one request at a time, so it is not used here), an optional answer-limit field of the card for MMLU-Pro and GPQA (this card does not set it, the limit stays 48 tokens), a choice of token counter, an API-key header and an HTTP time-out setting; the fixed question lists (files dated 18 and 19 Oct 2026), the answer limits, the sampling and the seed are unchanged.

MMLU-Pro (400 questions), GPQA Diamond (198), IFEval (200), LiveCodeBench (40 tasks) and RULER at 8K and 32K tokens (40 tasks each) ran with thinking off (chat template switch enable_thinking false), the instruct sampling of the Qwen model card (temperature 0.7, top-p 0.8, top-k 20, min-p 0.0, presence penalty 1.5; the kit's server does not apply the last two), seed 42, one request at a time. MMLU-Pro and GPQA Diamond ran with an answer limit of 48 tokens, letter only; IFEval with 4,096 tokens; LiveCodeBench with 32,768 tokens; RULER with 128 tokens. GPQA 30 ran with thinking on (the kit's own defaults: temperature 0.6, top-p 0.95, top-k 20), an answer limit of 16,384 tokens and one question at a time. GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 rows), because the original is gated and no Hugging Face token was available.

Cut-off answers count as wrong: 0 in MMLU-Pro, 1 in GPQA Diamond, 0 in IFEval, 1 of the 40 in LiveCodeBench (limit 32,768 tokens; in addition 8 of the 40 answers held no code block and count as wrong; the median answer used 1,303 tokens), 0 in long text (the RULER tests) and 9 of the 30 GPQA answers with thinking (limit 16,384 tokens; the median answer used 5,371 tokens and 95.7 seconds). One MMLU-Pro answer held no letter and counts as wrong. RULER by task: variable tracking 0 of 10 at 8K and 2 of 10 at 32K; multi-hop question answering 9 of 10 and 3 of 10; multi-key retrieval 10 of 10 and 10 of 10; common-word extraction 10 of 10 and 10 of 10. Three of the 80 RULER prompts came out one token away from the target length by the tokenizer count (8,191, 8,193 and 32,767 tokens).

Foreign load: these tests have no guard; their quiet check lists the busiest other process every 10 seconds. MMLU-Pro, GPQA Diamond, IFEval, LiveCodeBench and RULER: 618 samples, 32 above 25 % of one core: the box's own indexing service in 25 samples (at most 178.0 %), a re-index script in 2 samples (at most 96.3 %), a file search in 2 samples (at most 100.0 %), a container runtime process in 1 sample (at most 50.0 %) and a git job in 2 samples (at most 220.0 %). GPQA 30 with thinking: 409 samples, 13 above 25 %: the box's own indexing service in 11 samples (at most 107.0 %), a re-index script in 1 sample (at most 96.9 %) and a file search in 1 sample (at most 100.0 %). Among them are the box's own indexing service, a re-index script and git jobs; I did not find out what started the file searches and the container runtime process. None of this was repeated or removed. The answer files of every test except LiveCodeBench are published. Not published: the quiet-check files of both runs (quiet-before.txt, quiet-after.txt and ruhe.csv), LiveCodeBench's results file (lcb.jsonl) with its code files (lcb/code, 71 files) and sandbox files (lcb/sandbox, 124 files), and the 80 RULER prompt files (ruler/prompts).

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts mmlupro --job tag-1910

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts gpqa --job tag-1910

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts ifeval --job tag-1910

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts lcb --job tag-1910

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts ruler --job tag-1910

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts ruler --job tag-1910

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti.json --parts think --job tag-1910

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. Measured 2026-11-11. Raw files: result of every step · log · contract.

Note from the run: The server was restarted for every size with the context set to that size (the card's own start script, which takes the size from an environment variable Q2_CTX; the command of each size is in the raw contract), and one prompt that fills about 95 % of the size was sent (prompts of 3,798 to 168,347 tokens, 96 answer tokens at most); the model had to answer. All seven sizes, 4K to 176,128 tokens, loaded and answered. 176,128 tokens is the context of the kit's own 2.5 bpw profile (kit README line 311, with images), so the largest size tested is the recipe's window; larger sizes were not tried. The memory number is the GPU memory in use on the whole card (the peak of the test): 11.33 GiB at 4K and 15.03 GiB at 176,128 tokens. The decode speed shown (28.3 tok/s) is that of one short reply of 74 tokens after a prompt of 168,347 tokens (the first token came after 329.1 seconds); it is not comparable with the decode speeds of the speed test, and the replies of the sizes differ in length (61 tokens at 4K). The start script's output for each size (server-c<size>.log, 7 files) is not published.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-default-5060ti-g1.json --parts fit --job mia25-16gb-1111

How it plays the Gauntlet

One solo race, season seed 21, one attempt, no help.

Game rank as of Nov 11, 2026. The rank is the order of the Gauntlet finish times in this memory class. Race page and raw files.

The race ran the recipe named on its race page, with the sampling and answer limit of its lane config. The engine build of the race is not recorded in the race files.

Result (result.json) Race log (log.jsonl) Model answers (io.jsonl) System prompt Exact command Verify output Model config Race page

Also made this possible

• Quant (EXL3 2.50 bpw, revision 0cd10912)turboderp@turboderp_ Hugging Face GitHub
• Engine (ExLlamaV3 1.6.0 and the EXL3 format)turboderp (turboderp-org)@turboderp_ GitHub Hugging Face
• Model (Qwen3.8-27B)QwenHugging Face @Alibaba_Qwen GitHub Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x RTX 5060 Ti 16 GB

System

Linux (we used the Linux start script)

Model

turboderp/Qwen3.8-27B-exl3 2.50bpw

Quant (EXL3 2.50 bpw, revision 0cd10912)

huggingface.co/turboderp/Qwen3.8-27B-exl3/tree/0cd1091272291d008e06976acb81e25946cdc767

Model (Qwen3.8-27B)

huggingface.co/Qwen/Qwen3.8-27B

Engine

ExLlamaV3 1.6.0 in Mia's kit (commit bbb11db), one RTX 5060 Ti 16 GB, 176,128 tokens of context, KV cache in 4 bits — the program that loads the model and answers requests · github.com/turboderp-org/exllamav3/releases/tag/v1.6.0Build and settings we used: ExLlamaV3 1.6.0 (Mia one-click kit bbb11db), MTP

Recipe (one-click kit, commit bbb11db)

github.com/MiaAI-Lab/Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install/tree/bbb11db4b7ff688a62841c68e0892fa099bc3362 by Mia (MiaAI-Lab)

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Put these settings into the recipe's .env file. One setting per line.

   Copy

   MODEL_DIR=Qwen3.8-27B-EXL3-2.5bpw
   CONTEXT_SIZE=176128
   CACHE_QUANT=4
   GPU_MEM_GB=14.7
   DRAFT=mtp
   PROFILE=2.5bpw-176k

3. Start the server with this command. It loads the model into memory.

   Copy

   CUDA_DEVICE_ORDER=PCI_BUS_ID \
     CUDA_VISIBLE_DEVICES=1 bash linux/start.sh \
     --no-harness \
     --no-pick

   What the parts mean

   • CUDA_DEVICE_ORDER=PCI_BUS_IDcounts the GPUs in slot order, the same order nvidia-smi shows
   • CUDA_VISIBLE_DEVICES=1which NVIDIA GPU the program may use: the ID of the GPU, its number or the UUID that nvidia-smi -L prints
   • MODEL_DIR=Qwen3.8-27B-EXL3-2.5bpwthe folder with the model files
   • CONTEXT_SIZE=176128the context window in tokens: how much text the model can see at once
   • DRAFT=mtpMTP (multi-token prediction): the model guesses several tokens ahead and checks them, which makes answers faster
   • CACHE_QUANT=4keeps the context memory (KV cache) in 4 bits, which uses less GPU memory
   • GPU_MEM_GB=14.7how much GPU memory, in GB, the kit plans with
   • PROFILE=2.5bpw-176kthe name of the kit profile these settings belong to; the name gives the bits per weight and the context size

ExLlamaV3 1.6.0 (Mia one-click kit bbb11db), MTP; CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=1 bash linux/start.sh --no-harness --no-pick (.env: MODEL_DIR=Qwen3.8-27B-EXL3-2.5bpw CONTEXT_SIZE=176128 CACHE_QUANT=4 GPU_MEM_GB=14.7 DRAFT=mtp PROFILE=2.5bpw-176k)

Mia's own benchmark, run unchanged

This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section on this page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.

Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-19. Version 2.50.0, commit 21a3511.

QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.

Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.

Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.

Note from the run: sparkDash 2.50.0 (@21a3511) on a client machine (never in the measured group); Quality: seed 1234, temperature 0, thinking off, a truncated answer counts as wrong; decode, prefill and tool eval send no seed; own row, never mixed with quality-v2; foreign load on the client Mac: secret scan from about 22:16 (I/O, one core) and a site build from about 22:38 (nice 19, one core). Same setup, earlier day: the sparkDash run is from 19 Oct 2026, three weeks before the speed and memory tests of 11 Nov 2026. The model name, the kit commit (bbb11db), the engine text, the start command (up to the start script's own name) and the context are the same in the register rows of both runs (checked field by field). sparkDash is Mia's own benchmark tool, run unchanged; its decode and prefill figures are not those of the speed test above and must not be read as one number with them. They use other prompts and settings: sparkDash decodes up to 400 tokens for one request with a short structured prompt (35 prompt tokens, one measurement, no sampling setting recorded in its decode file), the speed test decodes code and prose prompts of 512 tokens at the instruct sampling of the model card, as the median of two runs after a warm-up. sparkDash reports 94.3 tok/s for 1 request at once, the speed test 53.0 tok/s; I did not test what causes the difference. Foreign load: before the server started, the job's quiet check passed and the GPU was idle (2 MiB in use); no readings of other load on the box were kept during the run, which took about 29 minutes. In the quality part, 8 MMLU answers reached the limit of 1,024 tokens and count as wrong.

python3 tools/quality2/sparkdash_client.py tools/quality2/cards/qwen38-27b-exl3-2.5bpw-kit-dsh-5060ti.json <box> --client <client machine> --job tag-1910 --out $RUN

tool-eval-bench --base-url http://127.0.0.1:<port> --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/run-sparkdash-r2/tooleval/result.json --no-live