Home / 256 GB Mac

256 GB Mac · Mac Studio M5 Ultra 256 GB

Ash's recipe: Qwen3.8-Flash-Next 6-bit on a Mac Studio M5 Ultra (TensorFold 1.2.9)

Weights: MLX 6-bit, affine, group 32, with native MTP head (by TensorFold on Hugging Face, revision 45032a19)

Mac Studio M5 Ultra, 256 GB unified memory

Recipe (docs/recipes/qwen3.8-flash-next.md at v1.2.9)Ash HartX @ashxhartGitHubHugging FaceWebsite
TensorFold 1.2.9 official release archive macos-arm64 (tensorfold-native, tag v1.2.9), native MTP drafts, --name bench, --context 262144, --parallel 2 --vision --prompt-cache-gib 8commit TensorFold v1.2.9 (tag object 539dc9d6 -> commit 6b9de695104b6b8b30c20458da87ac06c9480289; release Nov 15, 2026 01:22 UTC); checkpoint TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP 45032a191 GPUmeasured 2026-11-16

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed, the author's method

Ash's own speed method first

Ash's method (pull requests 3227, 3271 and 3277 of TensorFold): the 16 prompts of tools/bench_prompts.json, 512 tokens, one stream, every prompt drafted and again with drafts off; our driver calls the stream function of the release's own tool. Server as on this page (thinking off). Ash's recipe lists one stream at 142 to 214 tok/s for the 262,144 window of this machine (recipe, M5 Ultra line), measured by the recipe's own tool on his machine; for the same recipe the 1.2.8 page of 15 Nov gives its figures. With 16 of 16 greedy and 16 of 16 sampled prompts the drafted and the undrafted replies had equal tokens. Our standard test, Speed v2, follows below: other prompts, sampling temperature 0.7, thinking off.

MethodDecodeDetailRaw files
Greedy (temperature 0, seed 1234), drafted, median of 16 prompts208.7 tok/swithdrawn drafts ("draft": false): 89.4result · command
Sampled (temperature 1.0, seed 1234 + i), drafted, median of 16 prompts174.7 tok/swithout drafts: 88.8result · command
Speed v2

How fast it runs

Decode
129.4tok/s
Decode, code
138.2tok/s
Decode, prose
120.7tok/s
500 words
≈ 5.6s
Prefill at 8K
4415.0tok/s
First token at 2K
0.6s
MeasurementResult
Decode129.4 tok/s
Words per second89.5 words/s
Characters per second599.3 chars/s
Decode, code prompt138.2 tok/s
Decode, prose prompt120.7 tok/s
Spread of the decode runs (CV)4.9 %
Spread of the code runs (CV)4.9 %
Spread of the prose runs (CV)0.5 %
Decode, window 1131.9 tok/s
Prefill at 2K3709.2 tok/s
Prefill at 8K4415.0 tok/s
Prefill at 32K4301.8 tok/s
Time to first token at 2K0.6 s
Time to first token at 8K1.9 s
Time to first token at 32K7.6 s
Draft acceptance rate49.3 %
Mean accepted length2.5 tokens
1 request at once (total)112.0 tok/s
2 at once (total)106.0 tok/s
4 at once (total)99.5 tok/s
8 at once (total)96.5 tok/s
1 request at once (per request)112.0 tok/s
2 at once (per request)72.6 tok/s
4 at once (per request)38.9 tok/s
8 at once (per request)20.9 tok/s
Server slots (2: --parallel 2, as in the recipe's M5 Ultra line)2 slots
Long answer, wall time29.8 s

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens4116.4 tok/s4.0 s130.4 tok/s116.8 tok/s
32K tokens4174.1 tok/s7.9 s132.8 tok/s120.2 tok/s
64K tokens3694.4 tok/s17.7 s128.6 tok/s123.3 tok/s
128K tokens3461.4 tok/s37.9 s125.2 tok/s119.2 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-16. Raw files: speed result · speed contract · log · long answer.

What ran. Ash's TensorFold 1.2.9, the official release for macOS (arm64, archive checksum 00600160, signature of the checksum file checked; no unmerged pull request), serving Qwen3.8-Flash-Next as the MLX 6-bit checkpoint with the MTP head (TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP, the same files as the 1.2.8 page), with the flags of the recipe's line for the M5 Ultra: --parallel 2, --vision and an 8 GiB prompt cache. For Flash Next on Metal, 1.2.9 changes nothing against 1.2.8 (the release notes name CUDA and HIP changes only), and it started without a fallback. Where this run differs from the recipe. (1) Window: 262,144 tokens instead of the recipe's 1,048,576. The engine plans 238.59 GiB for the 1,048,576 window and the load limit of the recipe's line (FLASHNEXT_LOAD_LIMIT_GB=240) is 223.52 GiB, so it does not load on this 256 GB machine. (2) Presence penalty: the speed test's instruct profile uses presence penalty 1.5 elsewhere; here it is 0 because Metal refuses any other value. (3) Concurrency: the server has 2 sessions, so the levels 4 and 8 at once are above its slot count and queue. The recipe lists one stream at 142 to 214 tok/s for the 262,144 window (137 to 216 at 1,048,576); the speed test here (other prompts, sampling temperature 0.7, thinking off) measures a decode between 116.8 and 138.2 tok/s depending on prompt type and context depth; the table before it shows the figure by Ash's own method. Two of the three speed runs were stopped by the foreign-load guard (a browser renderer above 25 % of one core; the second stop was not looked into); this page shows the third. The prefill at 64K and 128K in this run (3,694 and 3,461 tok/s) is lower than in the 1.2.8 night run (4,257 and 4,102). In repeat runs of the context steps on both versions (raw files not part of this page) the 128K prefill varied from run to run between 3,565 and 4,102 tok/s, with no difference between 1.2.8 and 1.2.9.

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run flashnext-mac-tf129-speed-1611-speed-r2)
python3 speed2.py --base-url http://127.0.0.1:56397/v1 --label flashnext-mac-tf129-speed-1611-speed-r2 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $HOME/bench/speed2 --contract $HOME/bench/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/Qwen3.8-Flash-Next-MLX-6bit-MTP/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 0.0} --sampling-source https://huggingface.co/Qwen/Qwen3.8-Flash-Next README.md lines 342 (Thinking) and 343 (Instruct/non-thinking), local copy not published (same as the other Flash Next cards); the checkpoint generation_config.json (1.0/20/0.95) = thinking default. Ash's recipe command sets --temperature 0 (docs/recipes/qwen3.8-flash-next.md:19), overridden per request by the card sampling, which the API allows. Seed 42 for speed/quality. || FORCED DEVIATION Nov 15, 2026: the model card's instruct presence_penalty 1.5 is set to 0.0, because the TensorFold 1.2.9 Metal engine (backend 'flashnext-zig') refuses nonzero values: server answer 'presence_penalty and frequency_penalty are not supported by backend flashnext-zig: send 0 or leave them out' (raw raw/korrektheit/presence-1.5-answer.txt), zig/src/server/server.zig:335 at v1.2.9 (same lines, checked Nov 16, 2026), docs/api.md:373 at v1.2.9 (same lines, checked Nov 16, 2026) ('other engines refuse nonzero values with 400; zero ... keep working'), open issue #3113. temperature 0.7, top_p 0.8, top_k 20, min_p 0 stay as on the model card. Thinking set unchanged (its presence_penalty is 0). --model bench --no-cache-prompt --server-log $HOME/bench/server.log --engine-version-file $HOME/bench/engine-version.txt --server-slots 2 --server-slots-source server start log of TensorFold 1.2.9 (--context 262144 --parallel 2 --vision --prompt-cache-gib 8), file $HOME/bench/server-start-ash.log line 8: 'info: Flash Next: 2 session slots (262144,262144 tokens), session budget 41.2 GiB; prompt cache 8.0 GiB, shared target forwards, every stream's drafts in one command buffer; a lone request on the engine's own rounds' (read Nov 16, 2026 00:10); explicit count from Ash's M5-Ultra line docs/recipes/qwen3.8-flash-next.md:137 (v1.2.9, unchanged); --parallel auto (default) would use the single-reply host (docs/recipes/qwen3.8-flash-next.md:98) --server-ctx 262144 --server-ctx-source https://github.com/ashhart/TensorFold/blob/v1.2.9/README.md serve flags table line 311: '--context N: Bound prompt plus reply tokens; Qwen3.8-27B defaults to 32,768 and takes up to 262,144'
Quality v2, thinking on

Quality with thinking on

This is a different test from the thinking-off tests and is never averaged with them. The model thinks before it answers. Server: the line of this page plus --thinking-budget 8192, which is our addition: the recipe sets no thinking budget. The requests carry no budget of their own. Sampling of the model card for thinking (temperature 1.0, top-p 0.95, top-k 20, presence penalty 0), seed 42, a limit of 16,384 tokens for thinking plus answer, 2 questions at a time; an answer cut off at the limit counts as wrong. The right-hand column is the thinking-off result of the same weights on the page of TensorFold 1.2.8 (15 Nov 2026, page), a separate test shown for comparison only. The tests ran with a copy of our quality tool that has four extra parts for thinking on; the questions and the scoring are the same as with thinking off. The GPQA file shows counts only, the text is left out because it quotes the questions. The machine was not free of other load during this run (browser, search and system processes, some above 50 % of one core), so the seconds per question are not a speed figure and are not shown. Not run with thinking on: the two long-text tests, the memory test.

TestResult, 95 % marginRightCut off, count as wrongMedian thinking plus answerFor comparison: thinking off, 1.2.8Raw files
MMLU-Pro
400 fixed questions
88.5 % ± 3.1 pts354 / 4000372 tokens69.5 %summary · counts · log · command
GPQA Diamond
198 questions
88.4 % ± 4.5 pts175 / 19808,198 tokens52.0 %summary · counts · log · command
IFEval
200 fixed prompts, strict
92.5 % ± 3.7 pts185 / 2000966 tokens86.5 %summary · counts · log · command
LiveCodeBench
40 newest tasks, the code runs
67.5 % ± 13.9 pts27 / 4018,480 tokens65.0 %summary · counts · log · command
Credits

Also made this possible

Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
Mac Studio M5 Ultra, 256 GB unified memory
System
macOS (the Mac above)
Model
Qwen/Qwen3.8-Flash-Next (qwen4_exp: Gated DeltaNet, sparse attention, MoE, hyper-connections, n-gram embeddings), TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP @45032a19 (MLX affine 6-bit, group 32, native MTP head)
Model files (MLX 6-bit with MTP head); Hugging Face organisation TensorFold, whose only listed member is ashxhart
huggingface.co/TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP
Engine
TensorFold 1.2.9 official release (macOS arm64); speed with thinking off, quality with thinking on — the program that loads the model and answers requests · github.com/ashhart/TensorFold/tree/v1.2.9
Recipe (docs/recipes/qwen3.8-flash-next.md at v1.2.9)
github.com/ashhart/TensorFold/blob/v1.2.9/docs/recipes/qwen3.8-flash-next.md by Ash Hart

Steps

  1. Get the recipe, the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

    env FLASHNEXT_LOAD_LIMIT_GB=240 tensorfold-native serve TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP \
      --name bench \
      --host 127.0.0.1 \
      --port 8888 \
      --context 262144 \
      --parallel 2 \
      --vision \
      --prompt-cache-gib 8 \
      --temperature 0 \
      --keep-warm 86400 \
      --no-update-check (recipe line docs/recipes/qwen3.8-flash-next.md lines 17 to 20 and the M5 Ultra line at lines 137 to 142, v1.2.9, unchanged since v1.2.8; the official release binary tensorfold-1.2.9-macos-arm64; context 262144 instead of the recipe's 1048576). For the thinking-on test the same line plus --thinking-budget 8192, which is our addition: the recipe sets no thinking budget.

    What the parts mean

    • --parallel 2how many requests it answers at the same time
    • --host 127.0.0.1who can connect: only this computer
    • --port 8888the network port the server listens on
    • --name benchthe model ID the server advertises to clients
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
TensorFold 1.2.9 official release archive macos-arm64 (tensorfold-native, tag v1.2.9), native MTP drafts, --name bench, --context 262144, --parallel 2 --vision --prompt-cache-gib 8; env FLASHNEXT_LOAD_LIMIT_GB=240 tensorfold-native serve TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP --name bench --host 127.0.0.1 --port 8888 --context 262144 --parallel 2 --vision --prompt-cache-gib 8 --temperature 0 --keep-warm 86400 --no-update-check (recipe line docs/recipes/qwen3.8-flash-next.md lines 17 to 20 and the M5 Ultra line at lines 137 to 142, v1.2.9, unchanged since v1.2.8; the official release binary tensorfold-1.2.9-macos-arm64; context 262144 instead of the recipe's 1048576). For the thinking-on test the same line plus --thinking-budget 8192, which is our addition: the recipe sets no thinking budget.

Context and sampling: 262,144 tokens (the recipe's line for the M5 Ultra names 1,048,576; see the deviation), 2 sessions (--parallel 2), prompt cache 8 GiB, vision on, MTP drafts. The speed test sends thinking off per request; sampling of the speed test: instruct profile of the model card with presence penalty 0, seed 42. The thinking-on quality test below uses a server started with the extra flag --thinking-budget 8192.. Hardware: Mac Studio M5 Ultra, 256 GB unified memory. All raw files of this setup: raw/v2/qwen38-flash-next-mac-mlx6bit-mtp-tensorfold-129-official-ash.