Ash's TensorFold recipe: Qwen3.8-27B (MLX 4-bit + DFlash2) on a Mac Studio

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-03. Raw files: speed result · speed contract · log · long answer.

I ran Ash Hart's recipe for this model (docs/recipes/qwen3.8-27b.md, line 13, the same at tag v1.2.1 and at main on 3 Nov 2026): tensorfold serve TensorFold/Qwen3.8-27B-MLX-4bit --drafter z-lab/Qwen3.8-27B-DFlash2 --name bench. Weights and drafter are at the revisions that the recipe page at main names in its Mac mini section (22d8d538 and 50307d4c, line 350; that section was added after the tag v1.2.1); their weight files match the Hugging Face hashes (sha256). The 4-bit weights are the TensorFold team's own conversion (MLX affine, 4 bits, group size 64), not a quant of mine. The engine is the official release archive tensorfold-1.2.1-macos-arm64 (sha256 fb75ab0d). Its binary tensorfold-native (sha256 f167738e) is byte for byte the binary in the 1.2.1 engine update of the desktop app TensorFold for Mac 1.0.0 (the app itself bundles engine 1.2.0). I did not install or start the app.

Differences from the author's setup, all named: I started the binary from the command line, not from the app window. The engine comes from the release archive, not from the Homebrew install that the README and the recipe's Mac mini section name (same version, 1.2.1); the archive's binary is called tensorfold-native, so I start tensorfold-native serve where the recipe writes tensorfold serve (the README's release-archive steps define tensorfold as tensorfold-native). I added --port 8888 (the default is 8080). The recipe sets no sampling, so each request carries the sampling of the Qwen model card. The recipe sets no window and no stream count either, so the engine's defaults apply: 32,768 tokens and up to 8 streams. A larger window (the model takes up to 262,144 tokens) would be another setup, so I did not use one.

Because a prompt and its answer must fit that window together, three parts of the speed test have no result. The 32K prefill: the server refused all 3 prompts of 32,768 tokens because they leave no room for a reply, so the part counts as failed. The long answer: its request asks for up to 32,768 answer tokens, as many as the whole window, and the server refused it, so its raw file is empty. The context steps at 32K, 64K and 128K: the prompt plus 512 answer tokens does not fit. These cells are missing, not zero. Everything else ran: decode, prefill at 2K and 8K, the 16K context step and the requests at once. I did not repeat the failed parts with another window.

The headline is the mean of the code prompt (170.94 tok/s) and the prose prompt (146.25 tok/s), each the median of two runs (spread 6.67 % and 11.06 %). Thinking was off in every request (the model thinks by default). The tool's cache check found no cached prompt tokens in any of the 45 requests it examined. The DFlash2 drafts were accepted at a rate of 23.5 %, with a mean accepted length of 4.53 tokens.

Requests at once: the server has 8 streams, so 1, 2, 4 and 8 requests all fit. The start log says that the DFlash2 drafts serve one of the 8 streams, so read the numbers for 2 to 8 requests with that in mind; I did not test how the drafts are shared between the streams. The total at 1 request (116.83 tok/s) is lower than the decode headline because that test counts the whole answer time, including the wait for the first token, and uses the prose prompt only.

The Mac: the GPU wired-memory limit stands at 245,760 MB (iogpu.wired_limit_mb); I read it and did not change it. My script paused three macOS background services (corespotlightd, mediaanalysisd, photoanalysisd) before the run and resumed them afterwards. A music app was open during the run. In the monitor's 23 samples, other processes used up to 75.5 % of one core (a macOS system service). The test's own load guard repeats a step when one foreign process stays above 25 % CPU for two samples in a row (a sample every 0.5 seconds). It repeated 2 of its 26 steps, once each after about 5 seconds of quiet: requests at once, level 8, round 1 (a macOS system service, up to 56.1 %) and the 16K prose run 1 (the macOS settings agent, up to 48.5 %). In the decode steps the music app's helper process showed up to 54 % of one core, and in the first 16K step (code prompt) up to 30 %; the guard did not repeat those steps, because it repeats a step only when one process stays above 25 % CPU for two samples in a row.

The recipe page also has the author's own speed table for an M5 Ultra (lines 66 to 73, commit d0cba3bf of 15 Oct 2026, an older engine than 1.2.1; raw prompts from the repository's docs, 100 greedy tokens): prefill 1,325 tok/s at 2K, 1,269 at 8K and 1,171 at 32K; decode with DFlash2 162 to 226 tok/s at 2K, 102 at 8K and 120 at 32K. My tests are others (a code and a prose prompt, sampled, 512 tokens), so the two are not like for like and I do not call any difference a deviation. For the record: my prefill is 1,867 tok/s at 2K and 1,833 at 8K; my decode is 170.9 tok/s (code) and 146.2 (prose); at 16K it is 131.7 and 122.5 tok/s with a prefill of 1,748 tok/s. I have not repeated the author's test with the author's prompts. The recipe also says that the Metal engine is qualified on M5 Max and M3 Ultra (line 3); the M5 Ultra appears in its speed table only.

python3 speed2.py --base-url http://127.0.0.1:61391/v1 --label qwen38-mac-0311-s-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $HOME/bench/speed2 --contract $HOME/bench/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/.cache/huggingface/hub/models--TensorFold--Qwen3.8-27B-MLX-4bit/snapshots/22d8d538154e0e5b6f8dcb1fcc74a60a0798104c/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source https://huggingface.co/Qwen/Qwen3.8-27B README.md lines 252 (Thinking Mode) and 253 (Instruct/non-thinking mode), local copy not published line 252-253 (same as the other Qwen3.8-27B cards); the server default (checkpoint generation_config.json: temperature 1.0, top_p 0.95, top_k 20) is overridden per request, which the API allows (api.md lines 174-179). Ash's command sets no sampling flag. Seed 42 for speed/quality. --model bench --no-cache-prompt --server-log $HOME/bench/server.log --engine-version-file $HOME/bench/engine-version.txt --server-slots 8 --server-slots-source engine start log, logs/server.log (probe start Nov 3, 2026 12:03): 'info: Qwen3.8-27B: 8 streams of 32768 tokens, DFlash2 drafts for one of them' (--parallel auto, the default, README v1.2.1 line 97/317: up to 8 streams as memory allows). The log line says DFlash2 drafts serve one of the 8 streams (engine design); read the concurrency numbers with that in mind --server-ctx 32768 --server-ctx-source https://github.com/ashhart/TensorFold/blob/v1.2.1/README.md serve flags table, line 'Qwen3.8-27B defaults to 32,768 and takes up to 262,144' (no --context in the recipe docs/recipes/qwen3.8-27b.md line 13); README at tag v1.2.1, line 282

How well it answers

Five of the seven quality tests are shown. Long text at 32K and LiveCodeBench are missing on purpose: the window of 32,768 tokens leaves no room for a 32K prompt or for the long answers of LiveCodeBench, so a number for them would measure the window, not the model.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-03.

Note from the run: MMLU-Pro and GPQA Diamond ran with an answer limit of 1,024 tokens instead of my usual 48 (the setting of this setup's test card, as on my other setups of 2 and 3 Nov); same questions, seed 42, the first letter of the answer counts. IFEval allowed 4,096 tokens per answer. The tests ran 8 questions at a time, the stream count of the server. GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 questions), because the original is gated and no token was available. 9 of the 30 GPQA answers with thinking hit the limit of 16,384 tokens and count as wrong; the median answer used 6,480 tokens and 158.2 seconds, with 8 questions running at a time. The answers file of this thinking test is not published: the site's filter for power units flagged a figure in the model's reasoning text (it was a longitude, not a power value). The row was counted from the complete file.

Long text at 32K is not shown: the server window is 32,768 tokens and a 32K prompt is longer than that after the chat template, so the server refused all 40 prompts (HTTP 400, context length exceeded; the first prompt had 32,780 tokens). The raw files of the long-text test (ruler.summary.json, ruler.jsonl and the contract) still hold this 32K part as 0 of 40, and the contract lists a value of 0.0 for it. Those zeros are refusals, not answers, so I did not register or show them. LiveCodeBench was not run: its answers may run up to 32,768 tokens, which cannot fit in a window of 32,768 tokens together with the prompt. The recipe page gives no quality numbers for this checkpoint (its lines 145 and 146 only say that release memory and quality comparisons are TBD), so these are my own tests.

In the monitor's samples, other processes used up to 47.9 % of one core during the instruction tests (a macOS metadata process) and up to 22.2 % during the thinking test (a macOS system tool); the three background services named in the speed note were paused for both runs and resumed afterwards.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-tf-1.2.1-official-dflash2-m5ultra-0311.json --parts mmlupro --job qwen38-mac-0311-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-tf-1.2.1-official-dflash2-m5ultra-0311.json --parts gpqa --job qwen38-mac-0311-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-tf-1.2.1-official-dflash2-m5ultra-0311.json --parts ifeval --job qwen38-mac-0311-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-tf-1.2.1-official-dflash2-m5ultra-0311.json --parts ruler --job qwen38-mac-0311-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-tf-1.2.1-official-dflash2-m5ultra-0311.json --parts think --job qwen38-mac-0311-q

Also made this possible

• ModelQwenHugging Face @Alibaba_Qwen GitHub Website
• Engine (TensorFold 1.2.1, official release for macOS arm64)Ash HartGitHub @ashxhart Hugging Face Website
• Model files (MLX 4-bit, revision 22d8d538)TensorFoldHugging Face @ashxhart GitHub Website
• Draft model (DFlash2, revision 50307d4c)z-lab (Zhijian Liu)@zhijianliu_ Hugging Face GitHub

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

Mac Studio M5 Ultra, 256 GB unified memory, one box (Darwin kernel 27.0.0)

System

macOS (the Mac above)

Model

Qwen/Qwen3.8-27B (dense 27B, Gated DeltaNet + attention), TensorFold/Qwen3.8-27B-MLX-4bit @22d8d538 (MLX affine 4-bit, group 64) + z-lab/Qwen3.8-27B-DFlash2 @50307d4c drafter

Model files (MLX 4-bit, revision 22d8d538)

huggingface.co/TensorFold/Qwen3.8-27B-MLX-4bit

Draft model (DFlash2, revision 50307d4c)

huggingface.co/z-lab/Qwen3.8-27B-DFlash2

Engine

TensorFold 1.2.1, the official macOS release (native Metal engine), MLX 4-bit weights with DFlash2 drafts, 32,768 tokens of context (the engine's default), up to 8 streams — the program that loads the model and answers requests · github.com/ashhart/TensorFold/releases/tag/v1.2.1

Recipe (docs/recipes/qwen3.8-27b.md, tag v1.2.1)

github.com/ashhart/TensorFold/blob/v1.2.1/docs/recipes/qwen3.8-27b.md by Ash Hart

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888.

   Copy

   tensorfold-native serve TensorFold/Qwen3.8-27B-MLX-4bit \
     --drafter z-lab/Qwen3.8-27B-DFlash2 \
     --name bench \
     --port 8888

   What the parts mean

   • --port 8888the network port the server listens on
   • --drafter z-lab/Qwen3.8-27B-DFlash2the draft model: it guesses the next tokens and the main model checks them; without it the model decodes plain
   • --name benchthe model ID the server advertises to clients

TensorFold 1.2.1 official release archive macos-arm64 (tensorfold-native, tag v1.2.1; same binary as the engine of TensorFold for Mac 1.0.0 after the 1.2.1 engine update), DFlash2 drafts, --name bench, --context default 32768, --parallel auto; tensorfold-native serve TensorFold/Qwen3.8-27B-MLX-4bit --drafter z-lab/Qwen3.8-27B-DFlash2 --name bench --port 8888 (Ash's recipe docs/recipes/qwen3.8-27b.md line 13; start script $HOME/bench/q38-ctl-official121-0311.sh)