Ash's recipe: Qwen3.8-Flash-Next 6-bit on a Mac Studio M5 Ultra (TensorFold 1.2.9)
Weights: MLX 6-bit, affine, group 32, with native MTP head (by TensorFold on Hugging Face, revision 45032a19)
Mac Studio M5 Ultra, 256 GB unified memory

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.
Ash's own speed method first
Ash's method (pull requests 3227, 3271 and 3277 of TensorFold): the 16 prompts of tools/bench_prompts.json, 512 tokens, one stream, every prompt drafted and again with drafts off; our driver calls the stream function of the release's own tool. Server as on this page (thinking off). Ash's recipe lists one stream at 142 to 214 tok/s for the 262,144 window of this machine (recipe, M5 Ultra line), measured by the recipe's own tool on his machine; for the same recipe the 1.2.8 page of 15 Nov gives its figures. With 16 of 16 greedy and 16 of 16 sampled prompts the drafted and the undrafted replies had equal tokens. Our standard test, Speed v2, follows below: other prompts, sampling temperature 0.7, thinking off.
How fast it runs
| Measurement | Result |
|---|---|
| Decode | 129.4 tok/s |
| Words per second | 89.5 words/s |
| Characters per second | 599.3 chars/s |
| Decode, code prompt | 138.2 tok/s |
| Decode, prose prompt | 120.7 tok/s |
| Spread of the decode runs (CV) | 4.9 % |
| Spread of the code runs (CV) | 4.9 % |
| Spread of the prose runs (CV) | 0.5 % |
| Decode, window 1 | 131.9 tok/s |
| Prefill at 2K | 3709.2 tok/s |
| Prefill at 8K | 4415.0 tok/s |
| Prefill at 32K | 4301.8 tok/s |
| Time to first token at 2K | 0.6 s |
| Time to first token at 8K | 1.9 s |
| Time to first token at 32K | 7.6 s |
| Draft acceptance rate | 49.3 % |
| Mean accepted length | 2.5 tokens |
| 1 request at once (total) | 112.0 tok/s |
| 2 at once (total) | 106.0 tok/s |
| 4 at once (total) | 99.5 tok/s |
| 8 at once (total) | 96.5 tok/s |
| 1 request at once (per request) | 112.0 tok/s |
| 2 at once (per request) | 72.6 tok/s |
| 4 at once (per request) | 38.9 tok/s |
| 8 at once (per request) | 20.9 tok/s |
| Server slots (2: --parallel 2, as in the recipe's M5 Ultra line) | 2 slots |
| Long answer, wall time | 29.8 s |
Long prompts
| Prompt length | Prefill | Time to first token | Decode, code prompt | Decode, prose prompt |
|---|---|---|---|---|
| 16K tokens | 4116.4 tok/s | 4.0 s | 130.4 tok/s | 116.8 tok/s |
| 32K tokens | 4174.1 tok/s | 7.9 s | 132.8 tok/s | 120.2 tok/s |
| 64K tokens | 3694.4 tok/s | 17.7 s | 128.6 tok/s | 123.3 tok/s |
| 128K tokens | 3461.4 tok/s | 37.9 s | 125.2 tok/s | 119.2 tok/s |
Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-16. Raw files: speed result · speed contract · log · long answer.
What ran. Ash's TensorFold 1.2.9, the official release for macOS (arm64, archive checksum 00600160, signature of the checksum file checked; no unmerged pull request), serving Qwen3.8-Flash-Next as the MLX 6-bit checkpoint with the MTP head (TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP, the same files as the 1.2.8 page), with the flags of the recipe's line for the M5 Ultra: --parallel 2, --vision and an 8 GiB prompt cache. For Flash Next on Metal, 1.2.9 changes nothing against 1.2.8 (the release notes name CUDA and HIP changes only), and it started without a fallback. Where this run differs from the recipe. (1) Window: 262,144 tokens instead of the recipe's 1,048,576. The engine plans 238.59 GiB for the 1,048,576 window and the load limit of the recipe's line (FLASHNEXT_LOAD_LIMIT_GB=240) is 223.52 GiB, so it does not load on this 256 GB machine. (2) Presence penalty: the speed test's instruct profile uses presence penalty 1.5 elsewhere; here it is 0 because Metal refuses any other value. (3) Concurrency: the server has 2 sessions, so the levels 4 and 8 at once are above its slot count and queue. The recipe lists one stream at 142 to 214 tok/s for the 262,144 window (137 to 216 at 1,048,576); the speed test here (other prompts, sampling temperature 0.7, thinking off) measures a decode between 116.8 and 138.2 tok/s depending on prompt type and context depth; the table before it shows the figure by Ash's own method. Two of the three speed runs were stopped by the foreign-load guard (a browser renderer above 25 % of one core; the second stop was not looked into); this page shows the third. The prefill at 64K and 128K in this run (3,694 and 3,461 tok/s) is lower than in the 1.2.8 night run (4,257 and 4,102). In repeat runs of the context steps on both versions (raw files not part of this page) the 128K prefill varied from run to run between 3,565 and 4,102 tok/s, with no difference between 1.2.8 and 1.2.9.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 speed2.py --base-url http://127.0.0.1:56397/v1 --label flashnext-mac-tf129-speed-1611-speed-r2 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $HOME/bench/speed2 --contract $HOME/bench/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/Qwen3.8-Flash-Next-MLX-6bit-MTP/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 0.0} --sampling-source https://huggingface.co/Qwen/Qwen3.8-Flash-Next README.md lines 342 (Thinking) and 343 (Instruct/non-thinking), local copy not published (same as the other Flash Next cards); the checkpoint generation_config.json (1.0/20/0.95) = thinking default. Ash's recipe command sets --temperature 0 (docs/recipes/qwen3.8-flash-next.md:19), overridden per request by the card sampling, which the API allows. Seed 42 for speed/quality. || FORCED DEVIATION Nov 15, 2026: the model card's instruct presence_penalty 1.5 is set to 0.0, because the TensorFold 1.2.9 Metal engine (backend 'flashnext-zig') refuses nonzero values: server answer 'presence_penalty and frequency_penalty are not supported by backend flashnext-zig: send 0 or leave them out' (raw raw/korrektheit/presence-1.5-answer.txt), zig/src/server/server.zig:335 at v1.2.9 (same lines, checked Nov 16, 2026), docs/api.md:373 at v1.2.9 (same lines, checked Nov 16, 2026) ('other engines refuse nonzero values with 400; zero ... keep working'), open issue #3113. temperature 0.7, top_p 0.8, top_k 20, min_p 0 stay as on the model card. Thinking set unchanged (its presence_penalty is 0). --model bench --no-cache-prompt --server-log $HOME/bench/server.log --engine-version-file $HOME/bench/engine-version.txt --server-slots 2 --server-slots-source server start log of TensorFold 1.2.9 (--context 262144 --parallel 2 --vision --prompt-cache-gib 8), file $HOME/bench/server-start-ash.log line 8: 'info: Flash Next: 2 session slots (262144,262144 tokens), session budget 41.2 GiB; prompt cache 8.0 GiB, shared target forwards, every stream's drafts in one command buffer; a lone request on the engine's own rounds' (read Nov 16, 2026 00:10); explicit count from Ash's M5-Ultra line docs/recipes/qwen3.8-flash-next.md:137 (v1.2.9, unchanged); --parallel auto (default) would use the single-reply host (docs/recipes/qwen3.8-flash-next.md:98) --server-ctx 262144 --server-ctx-source https://github.com/ashhart/TensorFold/blob/v1.2.9/README.md serve flags table line 311: '--context N: Bound prompt plus reply tokens; Qwen3.8-27B defaults to 32,768 and takes up to 262,144'Quality with thinking on
This is a different test from the thinking-off tests and is never averaged with them. The model thinks before it answers. Server: the line of this page plus --thinking-budget 8192, which is our addition: the recipe sets no thinking budget. The requests carry no budget of their own. Sampling of the model card for thinking (temperature 1.0, top-p 0.95, top-k 20, presence penalty 0), seed 42, a limit of 16,384 tokens for thinking plus answer, 2 questions at a time; an answer cut off at the limit counts as wrong. The right-hand column is the thinking-off result of the same weights on the page of TensorFold 1.2.8 (15 Nov 2026, page), a separate test shown for comparison only. The tests ran with a copy of our quality tool that has four extra parts for thinking on; the questions and the scoring are the same as with thinking off. The GPQA file shows counts only, the text is left out because it quotes the questions. The machine was not free of other load during this run (browser, search and system processes, some above 50 % of one core), so the seconds per question are not a speed figure and are not shown. Not run with thinking on: the two long-text tests, the memory test.
| Test | Result, 95 % margin | Right | Cut off, count as wrong | Median thinking plus answer | For comparison: thinking off, 1.2.8 | Raw files |
|---|---|---|---|---|---|---|
| MMLU-Pro 400 fixed questions | 88.5 % ± 3.1 pts | 354 / 400 | 0 | 372 tokens | 69.5 % | summary · counts · log · command |
| GPQA Diamond 198 questions | 88.4 % ± 4.5 pts | 175 / 198 | 0 | 8,198 tokens | 52.0 % | summary · counts · log · command |
| IFEval 200 fixed prompts, strict | 92.5 % ± 3.7 pts | 185 / 200 | 0 | 966 tokens | 86.5 % | summary · counts · log · command |
| LiveCodeBench 40 newest tasks, the code runs | 67.5 % ± 13.9 pts | 27 / 40 | 1 | 8,480 tokens | 65.0 % | summary · counts · log · command |
Also made this possible
- Engine (TensorFold 1.2.9, official release)
Ash Hart@ashxhart GitHub Hugging Face Website - Model
Qwen@Alibaba_Qwen Hugging Face GitHub Website - Model files (MLX 6-bit with MTP head); Hugging Face organisation TensorFold, whose only listed member is ashxhart
TensorFoldHugging Face @ashxhart GitHub Website
Run it yourself
What you need and how to start it, step by step. The results below come from exactly this setup.
What you need
- Hardware
- Mac Studio M5 Ultra, 256 GB unified memory
- System
- macOS (the Mac above)
- Model
- Qwen/Qwen3.8-Flash-Next (qwen4_exp: Gated DeltaNet, sparse attention, MoE, hyper-connections, n-gram embeddings), TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP @45032a19 (MLX affine 6-bit, group 32, native MTP head)
- Model files (MLX 6-bit with MTP head); Hugging Face organisation TensorFold, whose only listed member is ashxhart
- huggingface.co/TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP
- Engine
- TensorFold 1.2.9 official release (macOS arm64); speed with thinking off, quality with thinking on — the program that loads the model and answers requests · github.com/ashhart/TensorFold/tree/v1.2.9
- Recipe (docs/recipes/qwen3.8-flash-next.md at v1.2.9)
- github.com/ashhart/TensorFold/blob/v1.2.9/docs/recipes/qwen3.8-flash-next.md by Ash Hart
Steps
Get the recipe, the engine and the model files from the list above.
Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.
env FLASHNEXT_LOAD_LIMIT_GB=240 tensorfold-native serve TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP \ --name bench \ --host 127.0.0.1 \ --port 8888 \ --context 262144 \ --parallel 2 \ --vision \ --prompt-cache-gib 8 \ --temperature 0 \ --keep-warm 86400 \ --no-update-check (recipe line docs/recipes/qwen3.8-flash-next.md lines 17 to 20 and the M5 Ultra line at lines 137 to 142, v1.2.9, unchanged since v1.2.8; the official release binary tensorfold-1.2.9-macos-arm64; context 262144 instead of the recipe's 1048576). For the thinking-on test the same line plus --thinking-budget 8192, which is our addition: the recipe sets no thinking budget.What the parts mean
--parallel 2how many requests it answers at the same time--host 127.0.0.1who can connect: only this computer--port 8888the network port the server listens on--name benchthe model ID the server advertises to clients
Start text for this setupfor reproducing↓
You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.
TensorFold 1.2.9 official release archive macos-arm64 (tensorfold-native, tag v1.2.9), native MTP drafts, --name bench, --context 262144, --parallel 2 --vision --prompt-cache-gib 8; env FLASHNEXT_LOAD_LIMIT_GB=240 tensorfold-native serve TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP --name bench --host 127.0.0.1 --port 8888 --context 262144 --parallel 2 --vision --prompt-cache-gib 8 --temperature 0 --keep-warm 86400 --no-update-check (recipe line docs/recipes/qwen3.8-flash-next.md lines 17 to 20 and the M5 Ultra line at lines 137 to 142, v1.2.9, unchanged since v1.2.8; the official release binary tensorfold-1.2.9-macos-arm64; context 262144 instead of the recipe's 1048576). For the thinking-on test the same line plus --thinking-budget 8192, which is our addition: the recipe sets no thinking budget.Context and sampling: 262,144 tokens (the recipe's line for the M5 Ultra names 1,048,576; see the deviation), 2 sessions (--parallel 2), prompt cache 8 GiB, vision on, MTP drafts. The speed test sends thinking off per request; sampling of the speed test: instruct profile of the model card with presence penalty 0, seed 42. The thinking-on quality test below uses a server started with the extra flag --thinking-budget 8192.. Hardware: Mac Studio M5 Ultra, 256 GB unified memory. All raw files of this setup: raw/v2/qwen38-flash-next-mac-mlx6bit-mtp-tensorfold-129-official-ash.