Mia's AI Lab recipe: Qwen3.8-Flash-Next (TensorFold TP2)
2x NVIDIA DGX Spark class boxes (GB10, 'box-4' the head with rank 0 and the API, 'GB10 box' the worker with rank 1), 128 GB unified memory each, tensor parallel 2 over a direct CX7 cable (two rails; the link check before the run read 109.28 / 109.20 Gb/s), standing GPU clock cap 2200 MHz on both boxes (the default application clock of the GB10 is 2418 MHz). The NVFP4 checkpoint is about 123.6 GiB of weights (132.73 GB, 25 files), copied to both boxes; the recipe keeps 10 GiB of memory free on each box by design.

Overall score as of 27 Oct 2026. Scores are relative to the best run in the class today; a better model later moves older runs down.
How the overall score is calculated
| Part | This run | Best in class | Relative | Weight | Points |
|---|---|---|---|---|---|
| Speed, one request | decode 58.3 tok/s | best 70.6 tok/s | 82.7 % | 17.5 % | 14.5 |
| Speed, 8 at once | total 167.2 tok/s | best 167.2 tok/s | 100.0 % | 17.5 % | 17.5 |
| Quality | mean of 7 tests: 66.9 % | best mean 69.1 % | 96.8 % | 25 % | 24.2 |
| Browser check | 9.5 of 10 | 95.0 % | 5 % | 4.8 | |
| My rating | 3 of 10 | 30.0 % | 35 % | 10.5 | |
| Overall | 100 % | 71.4 |
The seven quality tests: MMLU-Pro 69.0 · GPQA 53.0 · IFEval 89.5 · LiveCodeBench 62.5 · long text 8K 70.0 · long text 32K 57.5 · GPQA 30 with thinking 66.7.
The speed part is half the decode speed with one request and half the total speed with 8 requests at once, each relative to the best run of this class; a run without a total for 8 requests at once gets the better of its decode speed alone and the mix with its highest measured total. Quality is relative to the best run of this class, the browser check and my rating are out of 10. The parts are rounded one by one. sparkDash, the memory test and speculative-decoding numbers are not part of it. How the score is made, and the assumption behind it.
What it builds as an agent
Task: Voxel world (3D scene, one HTML file). One attempt, no help. Counted: the final file counts.
The exact promptcopy it and try it yourself↓
This is the task prompt word for word, as we handed it to the agent harness. The harness adds its own system prompt and tools, and some harnesses add short context notes of their own. A run that goes past 90 minutes also gets a short “finish now” message, as the prompt says. To try it yourself, give it to your own coding agent: it needs a shell and a working folder.
Design and create the most impressive, creative, elaborate and detailed
voxel art world you are capable of. The theme and style are ENTIRELY your
choice — pick whatever world you can execute best and push it as far as
you can: scale, variety, detail, color, animation, atmosphere, life.
Surprise me.
Hard requirements:
- Everything ends up in ONE single self-contained HTML file (game.html in
the current working directory) that opens and runs directly in Chrome.
Use WebGL and whatever libraries you like, but embed everything —
no internet access at view time, no external files.
- The scene must present itself: include an automatic slow cinematic
camera tour (orbiting the world, occasionally zooming in on highlights)
that starts on load, plus mouse controls (orbit/zoom) for manual viewing.
- All user-facing text (UI, HUD, titles, descriptions, hints) must be in
ENGLISH, regardless of the language of any project files you encounter.
When finished, verify your HTML actually loads and renders without
console errors, then state DONE with the absolute path of game.html.
Do not ask questions.
How you work (same for every model):
- You work alone in the folder /work (this is the current working directory). Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh game.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- Keep a working /work/game.html at all times. If you are stopped, the last version that loaded without errors counts.
- JavaScript libraries available offline in the test browser, through an import map with these exact URLs: "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js", "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/", "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js", "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js", "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js". No other external resources.
- After the scene is built, set `window.scene` to the Three.js scene (if you use Three.js) and `window.ready = true`.Play it yourself
Open full screenAI-generated The page this setup built, running live in your browser: click into it and use the controls it shows itself (usually drag to look around, scroll to zoom). It needs WebGL and can be slow on a weak graphics card.
The browser check
- page loads without page error1 / 1
- window.ready === true within 30 s1 / 1
- no console errors (whole session)1 / 1
- frame is not blank1 / 1
- frame rate in software GL (rAF over 5 s)0.5 / 1
- world: scene size 0.5 (parts >= 80 or >= 20000 triangles; canvas fallback), visible text English if any 0.51 / 1
- automatic camera tour: the view changes on its own in two successive ~5 s windows (1 each)2 / 2
- animation in the world (nodes / instances / particles moving besides the camera)1 / 1
- OrbitControls: drag and wheel change the picture beyond the animation baseline1 / 1
Measured 2026-10-27. Raw files: check · run info · command · final choice · contract · sampling · screenshot · clip.mp4 · web.mp4.
Other processes used up to 100 % CPU during the run. Browser check 9.5 out of 10 (the voxel-world task VX): the page loads without a page error (ready after 6.73 s), has no console errors, is not blank and shows a world (a voxel archipelago, 191 parts, 104,032 triangles, English text); the camera tour runs in both windows, nodes and particles animate besides the camera, and drag and wheel work. The lost half point is the frame rate (17.1 frames a second in software rendering, no GPU; 20 give the full point). The agent (OpenCode 1.19.3) ran 34 minutes 23 seconds, one try, no help, and stopped on its own, long before the 'finish now' mark at 90 minutes; its answer used 99,855 tokens. 24 versions of the page were seen on the way, 16 loaded without errors; the last clean one counts (index.html, sha256 c284ae1b...). My rating of the world it built: 3 of 10 (rated in the browser, 27 Oct 2026). The showcase is one build task in a browser; it does not measure the model's strength, and the page says so.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
run_showcase.sh --harness opencode-limit-input --task VX --upstream http://127.0.0.1:18080/v1 --model Qwen3.8-Flash-Next --sampling-file $RUN/flashdual-2610-show-r4/show.in/sampling-show.json --out $RUN/flashdual-2610-show-r4/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 1048576How fast it runs
| Measurement | Result |
|---|---|
| Decode | 58.3 tok/s |
| Words per second | 38.4 words/s |
| Characters per second | 272.8 chars/s |
| Decode, code prompt | 60.9 tok/s |
| Decode, prose prompt | 55.7 tok/s |
| Spread of the decode runs (CV) | 2.0 % |
| Spread of the code runs (CV) | 0.4 % |
| Spread of the prose runs (CV) | 2.0 % |
| Decode, window 1 | 58.1 tok/s |
| Prefill at 2K | 3256.8 tok/s |
| Prefill at 8K | 3326.9 tok/s |
| Prefill at 32K | 3198.9 tok/s |
| Time to first token at 2K | 0.6 s |
| Time to first token at 8K | 2.5 s |
| Time to first token at 32K | 10.2 s |
| Draft acceptance rate | 49.7 % |
| Mean accepted length | 2.5 tokens |
| 1 request at once (total) | 54.5 tok/s |
| 2 at once (total) | 82.3 tok/s |
| 4 at once (total) | 124.6 tok/s |
| 8 at once (total) | 167.2 tok/s |
| 1 request at once (per request) | 54.5 tok/s |
| 2 at once (per request) | 42.2 tok/s |
| 4 at once (per request) | 32.8 tok/s |
| 8 at once (per request) | 23.0 tok/s |
| Server slots (from the recipe) | 16 slots |
| Long answer, wall time | 77.5 s |
Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-10-27. Raw files: speed result · speed contract · log · long answer.
Requests at once: the server has 16 slots (from the author's recipe). The speed test ran with the author's setup: Mia's AI Lab recipe MiaAI-Lab/Qwen3.8-Flash-Dual-DGX-Sparks-TensorFold at commit dc05c91 (26 Oct 2026), started with its own one command (./start.sh, no flags), which sets up both GB10 boxes and starts both ranks; context 1,048,576 tokens (the recipe default, config.sh line 111), 16 slots (line 117), MTP drafts with copy drafts (line 126), FP8 KV (line 138), vision on (line 142). Sampling from the Qwen model card (instruct temperature 0.7, top_p 0.8, top_k 20, presence penalty 1.5), thinking off per request, seed 42, no prompt cache (the engine offers none). Two things are named openly: the recipe's pinned GHCR image tag does not match the checked-out patches, so prepare.sh fell back to its documented local build (Zig 0.17.0, about 8 minutes, then copied to the worker; no recipe file changed); and both boxes run a German locale, where the recipe's free-memory check reads the German word for memory and stops, fixed with LC_ALL=C in our start wrapper (no recipe file changed; the one host intervention behind the setup score of 9). The worker's weights went by copy, not the README's NFS example (NFS needs sudo, which this run does not use); about 124 GiB of disk on the worker, no effect on the engine. Setup from zero to the first answer: 72 minutes, 62 of them the checkpoint download (39 MB/s on the shared line). The headline 58.32 tok/s is the median of the code prompt (60.94) and the prose prompt (55.71), two runs each (spread 0.39 % and 2.03 %), 512 tokens, one warm-up excluded. Drafts were accepted at a rate of 49.7 % (2.52 tokens per round). First token: 0.63 s at 2K, 2.46 s at 8K, 10.24 s at 32K prompt; prefill 3,257 / 3,327 / 3,199 tok/s at those sizes. At the same time: 54.5 / 82.3 / 124.6 / 167.2 tok/s total at 1 / 2 / 4 / 8 requests at once (each of the 8 gets 23.0); the recipe serves up to 16 requests at once. The sampled prose number sits under the author's headline for a reason the protocol explains: a greedy re-check with the author's own method (sparkDash DecodeBench, prose, one request, 400 tokens, temperature 0) read 71.42 tok/s, above the author's 67.4, so the setup is not the cause; the fixed Speed v2 test samples at the model card's 0.7. The author's own numbers (README lines 67 to 70, sparkDash, 7 Oct 2026, different method: 67.4 tok/s alone, 152.2 for 4 requests, 231.2 for 8) are kept separate and are not comparable with the fixed Speed v2 test.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 speed2.py --base-url http://127.0.0.1:34793/v1 --label flashdual-2610-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/flashdual-2610-speed-r0/flashdual-2610-speed-r0/speed2 --contract $RUN/flashdual-2610-speed-r0/flashdual-2610-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/.cache/huggingface/hub/models--nvidia--Qwen3.8-Flash-Next-NVFP4/snapshots/fc694b54fb0174e0913e6adf86691ef85a4ead47/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source Qwen/Qwen3.8-Flash-Next README.md:343 (Instruct/non-thinking) and :342 (Thinking), local copy not published; generation_config.json (1.0/20/0.95) = thinking default only, not used for instruct --model Qwen3.8-Flash-Next --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 16 --server-slots-source MiaAI-Lab/Qwen3.8-Flash-Dual-DGX-Sparks-TensorFold@dc05c91 scripts/config.sh:117 (PARALLEL=${PARALLEL:-16}, requests decoded together, 1 to 16); start.sh:71 (--parallel); README: 'Up to 16 requests at once'How well it answers
| Test | Result95 % margin | Right | Cut offcount as wrong | Raw files | |
|---|---|---|---|---|---|
| MMLU-Pro 400 fixed questions, letter only | 69.0 % ± 4.5 pts | 276 / 400 | 0 | answers · summary · log · contract | command |
| GPQA Diamond 198 questions, letter only | 53.0 % ± 6.9 pts | 105 / 198 | 0 | answers · summary · log · contract | command |
| IFEval 200 fixed prompts, strict | 89.5 % ± 4.3 pts | 179 / 200 | 1 | summary · log · contract | command |
| LiveCodeBench 40 newest tasks, the code runs | 62.5 % ± 14.4 pts | 25 / 40 | 1 | summary · log · contract | command |
| Long text 8K 4 tasks x 10 at 8192 tokens | 70.0 % ± 13.7 pts | 28 / 40 | 0 | log · contract · answers · summary | command |
| Long text 32K 4 tasks x 10 at 32768 tokens | 57.5 % ± 14.7 pts | 23 / 40 | 0 | log · contract · answers · summary | command |
| GPQA 30, thinking on 30 questions, limit 16,384 tokens | 66.7 % ± 16.0 pts median 5,168 tokens, 263.2 s per question | 20 / 30 | 9 | log · summary | command |
Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-27.
Note from the run: Quality ran on this card on 27 Oct 2026, all seven tests of the fixed method: MMLU-Pro 400, GPQA Diamond 198, IFEval 200, LiveCodeBench 40 (code runs), Ruler 8K and 32K, plus GPQA 30 with thinking (cap 16,384 tokens). Thinking off in all tests except the thinking test. Sampling is the maker's (instruct: temperature 0.7, top_p 0.8, top_k 20, presence penalty 1.5, from the Qwen model card), seed 42, the questions ran 16 at a time, the recipe's slot count (same questions, limits and seed 42 for every question; the thinking test's median time per question, 263 s, was measured under that load). A cut-off answer counts as wrong: 1 of the 40 LiveCodeBench answers was cut at its limit, 2 across the instruct blocks in total, and 9 of the 30 thinking answers. The error margin sits next to every number. These are my own tests; no other run is mixed into the numbers on this page.
Exact commands we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts mmlupro --job flashdual-2610bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts gpqa --job flashdual-2610bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts ifeval --job flashdual-2610bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts lcb --job flashdual-2610bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts ruler --job flashdual-2610bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts ruler --job flashdual-2610bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts think --job flashdual-2610How much context fits
| Context size | Prompttokens | Filled | First token | Prefilltok/s, as the server reports it | Decodetok/s | Peak memoryGiB | Result |
|---|---|---|---|---|---|---|---|
| 4K 4,096 tokens | 3,891 | 95.0 % | — | — | — | 72.71 | answered (stop) |
| 8K 8,192 tokens | 7,782 | 95.0 % | — | — | — | 72.99 | answered (stop) |
| 16K 16,384 tokens | 15,564 | 95.0 % | — | — | — | 73.38 | answered (stop) |
| 32K 32,768 tokens | 31,129 | 95.0 % | — | — | — | 73.92 | answered (stop) |
| 64K 65,536 tokens | 62,259 | 95.0 % | — | — | — | 74.91 | answered (stop) |
| 128K 131,072 tokens | 124,518 | 95.0 % | — | — | — | 76.70 | answered (stop) |
| 256K 262,144 tokens | 249,036 | 95.0 % | — | — | — | 80.03 | answered (stop) |
| 512K 524,288 tokens | 498,073 | 95.0 % | — | — | — | 83.13 | answered (stop) |
| 1024K 1,048,576 tokens | 996,147 | 95.0 % | — | — | — | 87.86 | answered (stop) |
Memory test. The recipe fixes the context (1,048,576 tokens), so the server is started once with the recipe's own command and reserves its memory up front. Then, for each size (4K, 8K, 16K ... up to the largest size tested), one prompt that fills about 95 % of that size is sent, and the model must answer. The largest size that answered counts; it is the highest size tested, not a limit of the setup. Decode speed at that size was not measured. Peak memory is, on each of the two boxes, the rise of used memory (MemTotal - MemAvailable) over the idle value before the step; the table and the number show the highest of the two boxes. Unified memory, so GPU and CPU share it. Measured 2026-10-27. Raw files: result of every step · log · contract.
Note from the run: fit memory per node, highest rise over idle: box-4 87.9 GiB, box-1 83.6 GiB The memory test climbed the fixed ladder up to the recipe's full window: the model answered at 1,048,576 tokens of context (the recipe's default), so nothing larger was tested. The peak rise over idle was 87.86 GiB at the largest step (the number includes the model weights). Decode speed at these sizes is not measured. The first fit run hit a limit of my own test tool, not of the recipe: the filler pool ran out at the 1,048,576-token step, so that prompt was only 67.58 % full and the model stopped on its own; the tool fix (the pool now repeats) and the rerun are the numbers on this page.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json --parts fit --job flashdual-2610Also made this possible
- Model
QwenHugging Face @Alibaba_Qwen GitHub Website - Engine (TensorFold zig-flashnext)
Ash Hart@ashxhart GitHub Hugging Face Website - NVFP4 checkpoint (Qwen3.8-Flash-Next-NVFP4)
NVIDIAHugging Face GitHub Website
Run it yourself
What you need and how to start it, step by step. The results below come from exactly this setup.
What you need
- Hardware
- 2x NVIDIA DGX Spark class boxes (GB10, 'box-4' the head with rank 0 and the API, 'GB10 box' the worker with rank 1), 128 GB unified memory each, tensor parallel 2 over a direct CX7 cable (two rails; the link check before the run read 109.28 / 109.20 Gb/s), standing GPU clock cap 2200 MHz on both boxes (the default application clock of the GB10 is 2418 MHz). The NVFP4 checkpoint is about 123.6 GiB of weights (132.73 GB, 25 files), copied to both boxes; the recipe keeps 10 GiB of memory free on each box by design.
- Model
- nvidia/Qwen3.8-Flash-Next-NVFP4 @fc694b54
- NVFP4 checkpoint (Qwen3.8-Flash-Next-NVFP4)
- huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4
- Engine
- The official NVFP4 checkpoint, TensorFold zig-flashnext (Zig 0.17.0) with MTP and copy drafts, tensor parallel 2 over a direct CX7 cable, 1,048,576 tokens of context, 16 streams — the program that loads the model and answers requests · github.com/ashhart/TensorFold/tree/zig-flashnext
- Recipe (two GB10 boxes)
- github.com/MiaAI-Lab/Qwen3.8-Flash-Dual-DGX-Sparks-TensorFold by Mia (MiaAI-Lab)
Steps
Get the recipe, the engine and the model files from the list above.
Start the server with this command in the recipe folder. It loads the model into memory.
./start.shWe used the recipe at commit dc05c91. Our run started it on the first DGX Spark (head); the second DGX Spark ran the worker. How the recipe starts the two DGX Sparks: see the README of the recipe, github.com/MiaAI-Lab/Qwen3.8-Flash-Dual-DGX-Sparks-TensorFold.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the start text exactly as our run recorded it, with the engine build in front, so anyone can repeat the result.
TensorFold zig-flashnext db281878 tensorfold-native (Zig 0.17.0, Ash Hart + contributors; Flash Next port by Mia's AI Lab), TP=2, MTP + copy drafts, in image tensorfold-qwen38fn:zig-db28187 (built locally by the recipe's prepare.sh, patches 0001-0012); ./start.sh ($HOME/flashdual-recipe @dc05c91, no flags; scripts/local.sh: WORKER=me@127.0.0.1, WORKER_WEIGHTS=copy; head box-4 rank 0, worker GB10 box rank 1; via spark-2x-fn-dual.sh start)Context and sampling: 1,048,576 tokens (the recipe's CONTEXT, config.sh line 111 at dc05c91, YaRN x4 past 262,144), 16 slots, FP8 KV, MTP drafts with copy drafts, vision on, thinking on by default (off per request in the tests), seed 42, no prompt cache. The engine sized the KV pool at 23.81 GiB (the author names 26.5) because these boxes had about 110 GiB free at the start instead of the 113 the recipe expects; both boxes keep 10 GiB free by design. Sampling as the Qwen model card: instruct temperature 0.7 / top_p 0.8 / top_k 20 with presence penalty 1.5; thinking temperature 1.0 / top_p 0.95 / top_k 20. Harness for the build test: OpenCode 1.19.3 with the fixed launcher (the maker names no harness that runs locally on the box), headless, in a container on the worker GB10 box, reaching the server through a check proxy. A known recipe issue is named openly (the recipe's issue 7): a NoRoom error takes both ranks down when other work shrinks the reserve; it was not worked around for this run.. Hardware: 2x NVIDIA DGX Spark class boxes (GB10, 'box-4' the head with rank 0 and the API, 'GB10 box' the worker with rank 1), 128 GB unified memory each, tensor parallel 2 over a direct CX7 cable (two rails; the link check before the run read 109.28 / 109.20 Gb/s), standing GPU clock cap 2200 MHz on both boxes (the default application clock of the GB10 is 2418 MHz). The NVFP4 checkpoint is about 123.6 GiB of weights (132.73 GB, 25 files), copied to both boxes; the recipe keeps 10 GiB of memory free on each box by design.. All raw files of this setup: raw/v2/qwen38-flashnext-2sparks-mia-dual-tf.
Mia's own benchmark, run unchanged
This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section on this page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.
| Test | Result95 % margin | Right | Cut offcount as wrong | Raw files | |
|---|---|---|---|---|---|
| Quality QA 150 questions | 89.3 % ± 5.0 pts | 134 / 150 | 0 | answers · summary | command |
| Quality MMLU 285 questions | 89.8 % ± 3.5 pts | 256 / 285 sparkDash's own screen counts 258 right | 12 | answers · summary | command |
| Tool Eval Quick 15 scenarios, tool use | 100 / 100 | 15 / 15 passed | — | tool eval result · tool eval report · tool eval command · summary | command |
| Decode, 1 request at once first token 100 ms | 175.3 tok/s | — | — | decode result · summary | command |
| Prefill at 4K first token 1.3 s | 3526.6 tok/s | — | — | prefill result · summary | command |
| Prefill at 8K first token 2.5 s | 3437.7 tok/s | — | — | prefill result · summary | command |
| Prefill at 16K first token 5.0 s | 3393.0 tok/s | — | — | prefill result · summary | command |
| Prefill at 32K first token 10.1 s | 3298.2 tok/s | — | — | prefill result · summary | command |
Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-27. Version 2.50.0, commit 21a3511.
QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).
MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).
Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.
Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.
Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.
Note from the run: sparkDash 2.50.0 (@21a3511) ran its own tests on 27 Oct 2026 (job flashdual-2610-sd): its QA set 89.33 % (0 of 150 answers cut) and its MMLU set 89.82 % (12 cut), temperature 0, thinking off, a cut-off answer counts as wrong. Its own column, never mixed into Speed v2, Quality v2 or the ranking. The sparkDash client ran on another machine, through a relay, and was never part of the measured group.
Exact commands we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 tools/quality2/sparkdash_client.py tools/quality2/cards/qwen38-flashnext-nvfp4-zig-2x-dc05c91.json <box> --client <client machine> --job flashdual-2610-sd --out $RUNtool-eval-bench --base-url http://127.0.0.1:<port> --model Qwen3.8-Flash-Next --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/flashdual-2610-sd-sparkdash-r0/tooleval/result.json --no-live