Ash's TensorFold 1.2.7 on a Mac Studio: Qwen3.8-27B (MLX 4-bit, DFlash2 drafts)

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-11. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 8 slots (the server's own choice with --parallel auto, from its start line). What ran. The speed test ran TensorFold 1.2.7 as the official release archive for macOS (tensorfold-1.2.7-macos-arm64.tar.gz, published 11 Nov 2026, 17:18 UTC by the repository owner, sha256 b8e4c5e4, the digest GitHub lists for that file; the binary tensorfold-native inside has the sha256 69430b30, recorded with the run; no patch) and Ash's recipe of docs/recipes/qwen3.8-27b.md lines 19 to 21: the checkpoint TensorFold/Qwen3.8-27B-MLX-4bit at revision 22d8d538 and the drafter z-lab/Qwen3.8-27B-DFlash2 at revision 50307d4c with --drafter (README line 323: Qwen3.8-27B uses DFlash2 through that flag). I added --host 127.0.0.1, --port 8888, --keep-warm 86400, --no-update-check and, for this page, --context 262144 (see Window and streams); the exact command is above and in the start text. The recipe's text says that the Metal engine is 'qualified on M5 Max and M3 Ultra' (line 5); this page is a measurement on an M5 Ultra, and I show no number of the docs next to ours.

Window and streams. The window is 262,144 tokens, set with --context 262144; the recipe sets none, the server's default is 32,768 tokens (README line 307) and the model's own window is 262,144 tokens. I need the large window for the long-prompt steps up to 128K, for long text at 32K and for LiveCodeBench (answers of up to 32,768 tokens). This is my deviation from the recipe's command, and every number on this page was measured with it. The server chose the streams itself (no --parallel was passed; its default is auto, README lines 104 and 345): its start line reads '8 streams of 262144 tokens, DFlash2 drafts for 8 streams', and GET /v1/models reports a context of 262144 (speed result). In the published speed2.json the field server_slots_origin reads 'recipe': that is the tool's label for a slot count that is passed to it with --server-slots (here the value read from the server's start line, see server_slots_source), as against a count that the server itself reports through its API (server_slots_reported is empty here). The long-answer test asks for one reply of up to 32,768 tokens; the reply had 8,083 tokens and ended with a stop after 36.1 seconds.

Long prompts. All four steps (16K, 32K, 64K and 128K) fit the window and were measured at their exact length (the prompt has exactly that many tokens, counted by the server). They are part of the speed run itself (raw folder qwen38-mac-tf127-1111-s-speed-r0; there is no supplementary run on this page). The table of long prompts shows: 16K: 146.1 tok/s (code) and 133.4 tok/s (prose), prefill 1792.6 tok/s, first token after 9.1 s; 32K: 146.2 tok/s (code) and 127.4 tok/s (prose), prefill 1646.0 tok/s, first token after 19.9 s; 64K: 122.9 tok/s (code) and 109.4 tok/s (prose), prefill 1419.8 tok/s, first token after 46.2 s; 128K: 94.3 tok/s (code) and 90.9 tok/s (prose), prefill 1071.9 tok/s, first token after 122.3 s. The prefill series of this card has three sizes, 2K, 8K and 32K (all three are rows of the speed table); the tiles show the prefill at 8K and the first token at 2K.

Requests at once. The server has 8 streams, so all four levels (1, 2, 4 and 8 at once) are inside the streams. The total rises from 122.3 tok/s with 1 request to 204.8 tok/s with 8 requests, while the speed of one request falls from 122.3 to 26.9 tok/s. The words and characters per second come from the four decode runs.

The headline 169.4 tok/s is the mean of the code prompt (183.5) and the prose prompt (155.4), medians of two runs each (spread (CV) of 2.2 % and 6.8 %). The code prompt has 1,257 and 1,260 tokens and the prose prompt 1,352 and 1,356 tokens, and every reply is 512 tokens long and ends at the limit. Drafts were accepted at a rate of 23.8 % (1,581 of 6,640 drafted tokens, mean accepted length 4.4 tokens).

Sampling and a first attempt. Sampling is the Qwen model card's setting for replies without thinking (temperature 0.7, top_p 0.8, top_k 20, min_p 0; model card lines 252 and 253), sent with the request flag for no thinking and seed 42, with presence penalty 0 instead of the card's 1.5. That is a forced deviation: my first speed attempt on 11 Nov at 22:31, with the card's 1.5, failed before any measurement because the engine answered every request with HTTP 400 (docs/api.md line 284: the penalty is applied by Qwen3.8-27B and Qwen3.5-MoE on CUDA, and other engines refuse nonzero values with 400; line 431 names --instruct-sampling presence_penalty=0 as the way to keep the card's temperature, top_p and top_k without the penalty). That attempt has no number: its folder is not published, and it is not in the register. The chat template thinks by default and the command passes no sampling flag, so the server's own default is temperature 1.0, top_p 0.95, top_k 20.

Foreign load. The measuring tool put three macOS indexing processes on hold (media analysis, photo analysis, Spotlight's core service) for the run and resumed all three at its end. Its load guard (any other process above 25 % of a core for two samples in a row, sampled twice a second; the control session's own tool, the desktop shell and the dictation helper are on its ignore list) threw away and repeated 9 tries in 7 of the 38 steps, each after the machine was quiet again (the repeat waited 5 to 15 seconds); no step was invalid and the run was not stopped. The other processes that caused a try to be thrown away were: Browser Helper (Aperitif Renderer), PerfPowerServices, Python, ReportCrashService, Spotify Helper (Renderer), duetexpertd, mds_stores. In the kept tries of 6 steps the busiest other process the guard recorded was a web browser's page-renderer helper at 106 to 128 % of a core in its worst sample; in one step it was a music player's helper (94 %), and in the other steps a system process of macOS or its developer tools (at most 109 %). The quiet check of the speed run (one sample about every 10 seconds) took 175 samples, 26 of them above 25 % of a core; the highest was a macOS metadata worker at 360.6 % of a core. Not published: the guard's sample file, the quiet-check table and the single requests of the speed run.

python3 speed2.py --base-url http://127.0.0.1:59632/v1 --label qwen38-mac-tf127-1111-s-speed-r0 --prompts $BENCH/repos/<speed-test-tool>/prompts-v2.json --output $HOME/bench/speed2 --contract $HOME/bench/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/tf126-weights-1111/TensorFold_Qwen3.8-27B-MLX-4bit/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 0.0} --sampling-source https://huggingface.co/Qwen/Qwen3.8-27B README.md lines 252 (Thinking Mode) and 253 (Instruct/non-thinking mode), local copy not published line 252-253 (same as the other Qwen3.8-27B cards); the server default (checkpoint generation_config.json: temperature 1.0, top_p 0.95, top_k 20) is overridden per request, which the API allows (docs/api.md@v1.2.7 line 283: the request fields temperature, top_p, top_k and min_p are sampling overrides; line 401: a request's own fields win over the defaults). Ash's command sets no sampling flag. Seed 42 for speed/quality. || FORCED DEVIATION Nov 11, 2026 22:34 (first run of this card, job qwen38-mac-tf127-1111-s, failed at 22:31:57 with HTTP 400 on every request): the model card's instruct presence_penalty 1.5 is set to 0.0 here, because the TensorFold 1.2.7 Metal engine (backend 'qwen27') applies no penalties and refuses nonzero values: docs/api.md@v1.2.7 line 284 ('Applied by Qwen3.8-27B and Qwen3.5-MoE on CUDA; other engines refuse nonzero values with 400; zero, null and omitted values keep working'), zig/src/server/server.zig@v1.2.7 lines 329-330 (refusal text: presence_penalty and frequency_penalty are not supported by backend 'qwen27': send 0 or leave them out); docs/api.md@v1.2.7 line 431 names `--instruct-sampling presence_penalty=0` as the way to keep the card's temperature, top_p and top_k without the penalty. temperature 0.7, top_p 0.8, top_k 20, min_p 0 stay as on the model card. The Nov 3 card (engine 1.2.1) sent 1.5; whether 1.2.1 applied it is to verify. Thinking set unchanged (its presence_penalty was already 0). --model bench --no-cache-prompt --server-log $HOME/bench/server.log --engine-version-file $HOME/bench/engine-version.txt --server-slots 8 --server-slots-source server start log (1.2.7, --parallel auto, --context 262144), file $HOME/bench/server-vorstart-*.log line 7: 'info: Qwen3.8-27B: 8 streams of 262144 tokens, DFlash2 drafts for 8 streams' (read Nov 11, 2026 by the chain before the speed2 run, as this card's note required; the 1.2.6-card PLAN value 8 replaced) --server-ctx 262144 --server-ctx-source https://github.com/ashhart/TensorFold/blob/v1.2.7/README.md serve flags table line 307: '--context N: Bound prompt plus reply tokens; Qwen3.8-27B defaults to 32,768 and takes up to 262,144'

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-11.

Note from the run: MMLU-Pro and GPQA Diamond ran with an answer limit of 1,024 tokens (the tool's default is 48; the card of this setup sets 1,024; thinking off; the questions, the seed 42 and the first-letter extraction are the usual ones), IFEval with 4,096 tokens (prompt-level strict), long text with 128, LiveCodeBench with 32,768 and GPQA 30 with thinking with 16,384. Cut off at their limits: 0 of 400 MMLU-Pro, 0 of 198 GPQA and 1 of 200 IFEval answers and none in the long-text tests; 9 of the 30 thinking answers were cut off at the limit. A cut-off answer counts as wrong. The questions of each test ran 8 at a time, the 8 streams of the server (same questions, limits and seed for every question), so the median time per question of the thinking test was measured under that load: it is not a speed value. GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 questions), because the original is gated and no Hugging Face token was available.

Long text at 8K and 32K, 4 tasks of 10 each, scored strictly: multi-key retrieval 10 of 10 and 10 of 10, variable tracking 6 of 10 and 4 of 10, common-word extraction 9 of 10 and 9 of 10, multi-hop question answering 8 of 10 and 4 of 10. A variable-tracking task counts only when all variables are named (mean fraction 0.7 at 8K and 0.6 at 32K), and a multi-hop answer counts when the expected answer, normalised (lower case, no punctuation or articles), appears in the reply as a run of words. 4 of the 80 prompts are one token away from the exact length (8,191, 8,193, 32,767, 32,767 tokens; the tool marks them not exact).

LiveCodeBench: the 40 newest tasks of the set (contest dates 2025-03-15 to 2025-04-06), pass@1, the code runs in a sandbox without network. 18 of 40 passed. Of the 22 that failed, 4 were cut off at 32,768 tokens and count as wrong and 7 had no code in the reply. By difficulty, 5 of 9 easy, 6 of 11 medium and 7 of 20 hard tasks passed (counted from the answer file, which the site does not publish).

Thinking test (GPQA 30, thinking on): 30 of the GPQA questions, with the thinking-mode setting of the Qwen model card (temperature 1.0, top_p 0.95, top_k 20, min_p 0, presence penalty 0; model card line 252), seed 42, a limit of 16,384 tokens. 9 of the 30 were still thinking at the limit and count as wrong; of the 21 answers that finished, 20 are right. 66.7 % is the score at this limit. The median of 6,480 tokens and 151.3 seconds per question covers all 30 questions, including the 9 at the limit.

Sampling. The tests with thinking off sent the Qwen model card's setting for replies without thinking (temperature 0.7, top_p 0.8, top_k 20, min_p 0) with the request flag for no thinking and presence penalty 0 instead of the card's 1.5 (the engine refuses a nonzero penalty with HTTP 400, docs/api.md line 284; see the speed notes); every request of the quality tests sent cache_prompt false and seed 42. One run of each test, no repetition.

Foreign load. These tests have no load guard; their quiet check lists the busiest other process about every 10 seconds, and the measuring tool had the same three indexing processes on hold. MMLU-Pro, GPQA Diamond, IFEval and long text: 146 samples, 12 above 25 % of a core, up to 170.3 %; GPQA 30 with thinking: 82 samples, 6 above 25 %, up to 92.7 %; LiveCodeBench: 93 samples, 8 above 25 %, up to 100.0 %. Of these 26 samples 5 show a web browser's page-renderer helper, 3 the virtual machine of the code sandbox of the LiveCodeBench test itself, 2 the controlling Claude Code session, 2 a Python process of the developer tools, 1 a music player's helper, 1 the network extension of a VPN application and 12 macOS system processes. Nothing was repeated. The quiet check before each part found no other process above its limit; the one after the LiveCodeBench part found one, a macOS metadata worker (mds_stores) at 37 % of a core. Not published: the quiet-check tables, the LiveCodeBench answers, program and sandbox folders, the answers of the thinking test and the prompt files of the long-text test; the other summaries, logs, contracts and answer files that the site's filter lets through are published.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts mmlupro --job qwen38-mac-tf127-1111-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts gpqa --job qwen38-mac-tf127-1111-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts ifeval --job qwen38-mac-tf127-1111-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts lcb --job qwen38-mac-tf127-1111-lcb

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts ruler --job qwen38-mac-tf127-1111-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts ruler --job qwen38-mac-tf127-1111-q

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts think --job qwen38-mac-tf127-1111-q

How much context fits

Memory test. The context is fixed for the whole test (262,144 tokens, set with --context; the recipe sets none, see the notes on the window), so the server is started once with the command of this page and reserves its memory up front. Then, for each size (4K, 8K, 16K ... up to the largest size tested), one prompt that fills about 95 % of that size is sent, and the model must answer. The largest size that answered counts; it is the highest size tested, not a limit of the setup. Decode speed at that size was not measured. Peak memory is the rise of used memory over the idle value before each server start (unified memory, so GPU and CPU share it). Measured 2026-11-11 and 2026-11-12. Raw files: result of every step · log · contract.

Note from the run: fit memory per node, rise over idle (mac: rise of used memory over the idle value before each server start): at the highest level that answered, 262144, local 66.53 GiB = 68126 MiB (the value of the row q2_fit_peak_gib); highest value over all 7 levels: local 66.65 GiB = 68247 MiB at level 131072 (fit.json summary.peak_gib_by_node.local) The largest size tested is 262,144 tokens (256K), the model's own window and the top step of the series. It loaded and answered. The prompt of that step has 249,036 tokens (about 95 % of the window); the step of 4K ends at its answer limit of 96 tokens, all others with a stop. How it was measured: one server start with the card's command (window 262,144 tokens, no --parallel passed so the server's default auto applies, the same command as the speed and quality tests), then one prompt per size of about 95 % of the size, 96 tokens of answer. The server took 4 seconds to be up. The memory is the rise of used memory over the idle value before the start (unified memory: wired plus active plus compressed pages, read with vm_stat; idle 40.24 GiB). The tile 'Peak memory' shows the value of the largest size, 66.53 GiB at 256K; the highest value of the table is 66.65 GiB at 128K. The values are 64.53 GiB at 4K, 64.55 GiB at 8K, 64.58 GiB at 16K, 65.22 GiB at 32K, 65.83 GiB at 64K, 66.65 GiB at 128K, 66.53 GiB at 256K. They do not rise in line with the size of the prompt, and I did not look into why. Decode speed at these sizes is not measured (the memory test reports no timings). Not published: the server log of the memory test.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-mlx4bit-tf-1.2.7-official-dflash2-m5ultra-1111.json --parts fit --job qwen38-mac-tf127-1111-fit

Also made this possible

• ModelQwenHugging Face @Alibaba_Qwen GitHub Website
• Model files (MLX 4-bit, revision 22d8d538); Hugging Face organisation TensorFold, whose only listed member is ashxhartTensorFoldHugging Face @ashxhart GitHub Website
• Draft model (DFlash2, revision 50307d4c; the repository we loaded is z-lab's mirror of incoai/Qwen3.8-27B-DFlash2)Inco AI@inco_ai Hugging Face GitHub Website
• Speculative decoding method (DFlash) and the mirror we loadedz-lab@zhijianliu_ GitHub Hugging Face

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

Mac Studio M5 Ultra, 256 GB unified memory, one box (Darwin kernel 27.0.0)

System

macOS (the Mac above)

Model

Qwen/Qwen3.8-27B (dense 27B, Gated DeltaNet + attention), TensorFold/Qwen3.8-27B-MLX-4bit @22d8d538 (MLX affine 4-bit, group 64) + z-lab/Qwen3.8-27B-DFlash2 @50307d4c drafter

Model files (MLX 4-bit, revision 22d8d538); Hugging Face organisation TensorFold, whose only listed member is ashxhart

huggingface.co/TensorFold/Qwen3.8-27B-MLX-4bit

Draft model (DFlash2, revision 50307d4c; the repository we loaded is z-lab's mirror of incoai/Qwen3.8-27B-DFlash2)

huggingface.co/z-lab/Qwen3.8-27B-DFlash2

Engine

TensorFold 1.2.7, official release for macOS (arm64, Metal), the recipe's MLX 4-bit checkpoint with DFlash2 drafts, window 262,144 tokens, 8 streams chosen by the server — the program that loads the model and answers requests · github.com/ashhart/TensorFold/releases/tag/v1.2.7

Context window

262,144 tokens: how much text the model can see at once

Steps

1. Get the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   tensorfold-native serve <folder of TensorFold/Qwen3.8-27B-MLX-4bit> \
     --drafter <folder of z-lab/Qwen3.8-27B-DFlash2> \
     --name bench \
     --host 127.0.0.1 \
     --port 8888 \
     --context 262144 \
     --keep-warm 86400 \
     --no-update-check

   What the parts mean

   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --drafter <folderthe draft model: it guesses the next tokens and the main model checks them; without it the model decodes plain
   • --name benchthe model ID the server advertises to clients

TensorFold 1.2.7 official release archive macos-arm64 (tensorfold-native, tag v1.2.7), DFlash2 drafts, --name bench, --context 262144; no --parallel passed (the server's default auto applies); tensorfold-native serve <folder of TensorFold/Qwen3.8-27B-MLX-4bit> --drafter <folder of z-lab/Qwen3.8-27B-DFlash2> --name bench --host 127.0.0.1 --port 8888 --context 262144 --keep-warm 86400 --no-update-check (recipe: tensorfold pull TensorFold/Qwen3.8-27B-MLX-4bit; tensorfold pull z-lab/Qwen3.8-27B-DFlash2; tensorfold serve TensorFold/Qwen3.8-27B-MLX-4bit --drafter z-lab/Qwen3.8-27B-DFlash2 --name bench, docs/recipes/qwen3.8-27b.md lines 19 to 21. I used local copies of the same files in place of the Hugging Face names (the sha256 of all four files equals the Hugging Face file id, checked on 11 Nov 2026) and added --host 127.0.0.1, --port 8888, --context 262144 (the default is 32,768, README.md:307), --keep-warm 86400 and --no-update-check (README.md lines 326 and 364))