Ash's TensorFold recipe: Google Gemma 4 31B (MLX 4-bit + DFlash drafter) on one DGX Spark

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-10. Raw files: speed result · speed contract · log.

Requests at once: the server has 4 slots (from the author's recipe). C8 exceeds the slots: the extra requests wait in the queue, and they were measured anyway. The speed test ran with Ash Hart's recipe for this model (docs/recipes/gemma-4-native.md, lines 15 and 16; release v1.2.5): tensorfold-native serve with --port 8888, --parallel 4 and --drafter. The recipe is written for the native Metal build on a Mac (its title is 'Gemma 4 31B, native Metal'); the README of the release lists the same checkpoint for 'One Mac (M5 Ultra checked), or one GB10 on CUDA' (README, line 200), and the release notes name Gemma 4 31B 4-bit on a GB10 (lines 53 and 54). The weights are the community's MLX conversion of Google's checkpoint (mlx-community/gemma-4-31B-it-4bit, revision 696d436c; affine 4 bits, group size 64: 4 weight files, 17.18 GiB for all files of the repository, including the image tower, which is not loaded), not a quant of mine. The draft model is z-lab's DFlash for Gemma 4 31B (z-lab/gemma-4-31B-it-DFlash, revision eabd6483, 2.86 GiB), the drafter the recipe names; DFlash is a speculative decoding method that uses a lightweight block diffusion model to draft multiple tokens in parallel (z-lab's model card, line 20); the main model checks the drafted tokens. I downloaded both with hf download at those revisions, as the recipe's own commands do, and checked every file against the Hugging Face sizes and sha256 hashes; DGX Spark B got the same files by a copy from DGX Spark A, checked against the same hashes after the copy. The engine is the official release archive tensorfold-1.2.5-linux-aarch64.tar.gz (tag v1.2.5, sha256 1a0192b2; its binary tensorfold-native, sha256 4f7ed2ab), not built by me; it runs as a plain process, no container.

Which machine ran what: two identical DGX Sparks (same model of machine, same GPU clock cap, same engine file, same weights and drafter files) shared this work, because the quality tests are long at this decode rate. DGX Spark A ran this speed test, MMLU-Pro, GPQA Diamond and the two long-text tests (RULER). DGX Spark B ran IFEval, the memory test, the thinking test and LiveCodeBench. Every number on this page comes from one of the two machines; no number mixes them, and no test ran on both.

Differences from the author's recipe, all named: (1) A window of 147,456 tokens (--context 147456) instead of the default of 32,768 (recipe, lines 66 and 67). The context steps of 16K to 128K and the 32K prefill need a larger window: a prompt of 128K tokens plus its answer does not fit into 32,768 tokens. I first asked for the checkpoint's 262,144 with the recipe's --parallel 4: the engine refused to start ('--parallel 4 needs 86.80 GiB at a 262144-token window and the CUDA memory budget leaves 83.45 GiB after the weights: 3 streams fit'; a start attempt on DGX Spark A, log not published). I kept --parallel 4 and lowered the window to 147,456, the next multiple of 16,384 above the 131,584 tokens of the 128K step. This is a limit of this setup on one DGX Spark: the checkpoint's own window is larger than what runs here with 4 streams, and this page has no result above 144K tokens. (2) The server was started through a wrapper script of mine that checks the binary's hash, the version and the weights marker, evicts the weight files from the operating system's page cache with dd before the start (so the load starts from a clean cache), checks that the box is idle, and watches the server (it would stop it when free memory fell below 6 GiB or the chip became too hot; it never stopped a server in the runs of this page). The command it starts is the recipe's command plus --context 147456 and --name gemma-4-31b, with local folders from hf download --local-dir instead of the recipe's $HOME/models names, and the release's binary tensorfold-native directly. No --vision and no other flag, and no environment variable is set for the engine. (3) The release notes and the changelog describe an optional setting for Gemma 4 on CUDA, TF_GEMMA4_CHUNK_SOLO=2048 (release notes, line 53; changelog, line 100: a prompt pass takes 2,048 rows while no other stream decodes; the author reports a cold prompt's first token 4 to 5 % sooner at 4K, 16K and 29K tokens on a GB10 with the 31B 4-bit, 'Unset, nothing changes', and 1.2 GiB more for each stream's sliding rings). The recipe's command does not set it and I did not set it, so the prefill and first-token figures here are those of the default. (4) The recipe page gives its decode numbers for an M5 Ultra Mac on Metal: plain decode at about 51 tokens a second and 150 to 190 tokens a second for one stream with the drafter, 6 to 7 accepted tokens a round on chat and code and 2 on open-ended prose (lines 23 to 26); its qualification table (line 92) lists the pinned 31B 4-bit on an M5 Ultra with macOS 27, and lines 214 and 215 say that its measurements are the original pull request's receipts. It gives no figure for a DGX Spark. I measure on a GB10 with the model card's sampling, 512 tokens and separate code and prose prompts, so the author's figures and these are not comparable, and the page makes no statement about them. Text only: no vision, no tool use and no agent task is measured.

The headline 22.3 tok/s is the mean of the code prompt (24.9) and the prose prompt (19.7), medians of two runs each; the spread of the two code runs is 7.2 % and of the two prose runs 3.3 % (both under the 15 % limit of our tool). Thinking was off in every request, and the tool's cache check found no cached prompt tokens in any of the 60 requests it examined. The DFlash drafts were accepted at a rate of 26.7 % of the drafted tokens (1,194 of 4,465), 2.4 tokens per step on average. Requests at once: 1, 2, 4 and 8 requests ran together. The server has 4 slots, so with 8 requests 4 of them wait in the queue; the tool measured them anyway. The total rises from 18.5 tok/s with 1 request to 47.8 with 4 and 48.3 with 8, and the speed of one request falls from 18.5 to 8.3 tok/s with 8 requests; the 8-request figures include the time spent waiting for a slot, so they are not a measure of a setup with 8 slots, and the 1-request figure is measured over the whole request, so it is below the decode speed. Long prompts: the first token of the 128K-token prompt came after 350.4 s (prefill 374.0 tok/s), see the table.

Load on the machine (DGX Spark A): the load guard (a sample every half second, 7,456 samples) threw away one try of one of its 37 steps and repeated that step once: the first try of the second 64K-token code run was discarded because the operating system's package manager (apt) used up to 100 % of one core for 3 samples in a row; the guard waited 5 s for a quiet box and ran the step again, and the value on this page is from that second try. The guard's record of that second try names mstflint, a system tool that queries the network card's firmware, as the busiest other process, at 56.0 % of one core in its busiest sample (it ran in short bursts, never the same process above 25 % in two samples in a row, which is the guard's rule). The first-token time of that run, 124.3 s, lies within 0.5 s of the three other 64K runs (123.8 s to 124.0 s). In the other 36 steps the busiest other process stayed at or below 18.0 % of one core. The quiet check (one sample every 10 seconds, 372 samples) saw 4 samples above 25 % of one core, all between 08:51 and 08:52 on 10 Nov 2026: 3 of them mstflint and 1 the operating system's update check (apt-check); the highest was 72.7 %. The quiet logs before and after the run list no foreign problems. Not published: the quiet logs and the sampling file of this run (the site's filter holds back files of that kind by name), the load guard's sample file (it names a system service) and error file, and the test tool's server logs. So the foreign-load figures above can be checked only in part: the speed contract, with the summaries of the load guard and of the quiet check and its record of each step, is published, while the guard's sample file, the quiet logs and the sampling file are not. In the published raw files the source notes (the line numbers of the author's recipe, the names of our machines and folders, a few German words) were corrected or made neutral; no measured value was touched.

python3 speed2.py --base-url http://127.0.0.1:43173/v1 --label g4a-1010-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/g4a-1010-speed-r0/g4a-1010-speed-r0/speed2 --contract $RUN/g4a-1010-speed-r0/g4a-1010-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/gemma4-31b-1010/gemma-4-31b/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64, "min_p": 0.0} --sampling-source google/gemma-4-31B-it README 'Best Practices' (temperature 1.0, top_p 0.95, top_k 64, standardized for all use cases) = generation_config.json of mlx-community/gemma-4-31B-it-4bit @696d436c (temperature 1.0, top_k 64, top_p 0.95). No presence penalty. Seed 42 for speed/quality. --model gemma-4-31b --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 4 --server-slots-source gemma-4-native.md:15 and :19 (--parallel 4, caps four active requests; more wait for a free slot). C8 queues 4 requests by design (more requests than slots). --server-ctx 147456 --server-ctx-source recipe default 32768 (gemma-4-native.md:66-67); --context 147456 forced: 4 streams x 262144 tokens need 86.80 GiB according to the start message, the CUDA memory budget leaves 83.45 GiB after the weights (3 streams fit); --context 147456 is the next multiple of 16384 above the 131584 tokens (131072 prompt tokens + 512 answer tokens) of the 128K prefill test, --parallel 4 stays as in the recipe (start message of TensorFold 1.2.5 on one of our two identical DGX Sparks (machine A), probe of 10 Nov 2026, 08:19: '--parallel 4 needs 86.80 GiB at a 262144-token window and the CUDA memory budget leaves 83.45 GiB after the weights: 3 streams fit')

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-10.

Note from the run: MMLU-Pro (400 questions), GPQA Diamond (198), IFEval (200), LiveCodeBench (40 tasks) and RULER at 8K and 32K tokens (40 tasks each) ran with thinking off and the sampling of the model card (temperature 1.0, top-p 0.95, top-k 64), seed 42. Four requests ran at once (the server's 4 slots; each question is its own request with the same limits and seed). MMLU-Pro and GPQA Diamond ran with an answer limit of 1,024 tokens (the other pages use 48), letter only; IFEval with 4,096 tokens; LiveCodeBench with 32,768 tokens; RULER with 128 tokens. Most answers to the multiple-choice questions are a single letter (median 2 tokens). But with thinking off the model wrote a worked solution before its answer in some replies: 28 MMLU-Pro answers and 34 GPQA answers had no letter that the tool could read, and they count as wrong. The tool takes the letter when the reply starts with it or names it after a phrase such as 'answer is' or 'Answer:'; it does not guess. In 27 of the 28 MMLU-Pro replies and in 34 of the 34 GPQA replies the last line is a letter: 23 and 21 of them end with the line 'Answer (one letter): X', 4 and 13 with the letter alone; the tool does not read such an ending and I did not score them again, so the two scores are counted by the rule and the answer files show the rest (the GPQA answers are published; the MMLU-Pro answers are held back, see below). 5 MMLU-Pro answers and 5 GPQA answers were cut off at the 1,024-token limit (counted wrong, 10 in all); 0 IFEval answers and 1 LiveCodeBench answer were cut off at their limits, and the long-text tests had none cut off. RULER by task at 8K and 32K: multi-key retrieval 10 of 10 and 10 of 10; common-word extraction 10 of 10 and 10 of 10; multi-hop question answering 9 of 10 and 5 of 10; variable tracking 10 of 10 and 7 of 10. Two of the 40 prompts at 32K were one token off the target length (32,767 and 32,769 tokens instead of 32,768; the tool counts with the model's own tokenizer). IFEval is scored with the official checker of Google Research (instruction_following_eval, commit 3ef52111), strict, at the level of the prompt. LiveCodeBench: the 40 newest tasks of the tool's list (code_generation_lite, release test6; tasks dated 2025-03-15 to 2025-04-06), pass at 1, the code runs in a container without network access; no task was lost to a sandbox error or to a missing code block. The thinking test (GPQA 30) ran with thinking on and a limit of 16,384 tokens (thinking and answer together): 1 of the 30 answers was cut off at the limit and counts as wrong; the model thinks long: median 6,381 tokens and 329.3 s per question, with 4 questions running at once. A first attempt at the thinking test on the same machine (DGX Spark B, 10 Nov 2026) was stopped by me at 09:53 after 6 of the 30 questions were done, because it would not have finished in the time planned for it; the other 24 questions of that attempt are recorded as errors. I ran the whole test again from the start (12:02 to 12:47), all 30 questions, and that second run is the one on this page; the raw files of the first attempt are not part of this page. These quality tests have no foreign-load guard. Their sampling files (not published, see the speed notes) list the busiest other process every 10 seconds. DGX Spark A, MMLU-Pro and GPQA (112 samples): at most 19.1 % of one core, from an ssh login session. DGX Spark A, RULER (249 samples): 3 samples above 25 % of one core, all mstflint, the system tool that queries the network card's firmware, up to 71.1 %. DGX Spark B, IFEval (72 samples): at most 0.8 %. DGX Spark B, the thinking test (268 samples): 1 sample above 25 %, a system information script of the operating system, 69.2 %. DGX Spark B, LiveCodeBench (228 samples): 4 samples above 25 % of one core: 1 a system information script of the operating system (56.5 %) and 3 the code sandbox of the LiveCodeBench test itself. GPQA Diamond: the original is gated and no Hugging Face token was available, so the public mirror fingertap/GPQA-Diamond (198 rows) was used. Not published, because the site's filter held them back: the MMLU-Pro answers (mmlupro.jsonl; a model answer contains a number with a unit that the filter holds back), the IFEval answers (ifeval.jsonl; a model answer contains a word the filter reads as a machine name), and the LiveCodeBench results file and generations (lcb.jsonl, lcb.gen.jsonl; model code contains a word the filter reads as a hardware setting). The summaries and logs of all tests are published, and the numbers of the table were counted from the complete files. The quality tests ran on two identical DGX Sparks (see the speed notes): MMLU-Pro, GPQA and RULER on A; IFEval, the thinking test and LiveCodeBench on B. These are my own tests, and no other run of this setup is mixed into the numbers on this page.

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts mmlupro --job g4b-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts gpqa --job g4b-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts ifeval --job g4j-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts lcb --job g4k-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts ruler --job g4c-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts ruler --job g4c-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts think --job g4h-1010

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed at the largest size was not measured. Peak memory is the rise of used memory (MemTotal - MemAvailable) over the idle value before the step, on the one box. Unified memory, so GPU and CPU share it. Measured 2026-11-10. Raw files: result of every step · log · contract.

Note from the run: In this test the server was started again for each size, with its window set to that size (4,096 up to 147,456 tokens) and with the recipe's --parallel 4. Then one prompt of about 95 % of the window was sent, and the model had to answer. All seven sizes were answered (finish reason stop). The largest size tested is 147,456 tokens (a prompt of 140,083 tokens): it is the highest size I tested. The engine refused the checkpoint's 262,144-token window at start with 4 streams (message quoted in the speed notes: it needs 86.80 GiB, the budget leaves 83.45 GiB), and I did not test sizes between 147,456 and 262,144 tokens. So there is no result above 144K tokens. The memory number is the rise of the DGX Spark's used memory over its idle value before each server start (the GB10 has one pool of memory for CPU and GPU): the rise was 26.74 GiB with the 4K window and 87.29 GiB with the 144K window. The engine's start line in the speed run reads '4 streams at once, 15.23 GiB each at a 147456-token window', and its refusal message says that each stream's caches are allocated at load; the rise with the window is therefore the reservation of the caches of all 4 streams. Because the number is the rise of the whole machine, other programs that used or released memory at the same time are inside it. This engine reports no timings for this memory test, so the decode speed and the first-token columns are empty. This test ran on DGX Spark B (see the speed notes). The seven server logs of this test (one per size) are not published, because the site's filter held them back (they contain German text from our start script); fit.json, the test's output and its contract are published.

bash tools/quality2/measure_all.sh tools/quality2/cards/gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-1x-1010.json --parts fit --job g4i-1010

How it plays the Gauntlet

One solo race, season seed 21, one attempt, no help.

Game rank as of Nov 10, 2026. The rank is the order of the Gauntlet finish times in this memory class. Race page and raw files.

The race ran the recipe named on its race page, with the sampling and answer limit of its lane config. The engine build of the race is not recorded in the race files.

Result (result.json) Race log (log.jsonl) Model answers (io.jsonl) System prompt Exact command Verify output Model config Race page

Also made this possible

• Engine (TensorFold 1.2.5, official release)Ash Hart@ashxhart GitHub Hugging Face Website
• Model (Gemma 4 31B, built by Google DeepMind)GoogleHugging Face Website
• MLX 4-bit weights (revision 696d436c, converted from Google's checkpoint)MLX CommunityHugging Face
• Draft model (DFlash, revision eabd6483)z-lab (Zhijian Liu)@zhijianliu_ Hugging Face GitHub

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1 NVIDIA DGX Spark (GB10, 128 GB unified memory), GPU clock capped at 2200 MHz; two identical DGX Sparks took part, see the speed notes

Model

google/gemma-4-31B-it (dense 31B, 60 layers, 5 sliding : 1 global attention, vision tower not loaded), mlx-community/gemma-4-31B-it-4bit @696d436c (MLX affine 4-bit, group 64) + z-lab/gemma-4-31B-it-DFlash @eabd6483 drafter

Model (Gemma 4 31B, built by Google DeepMind)

huggingface.co/google/gemma-4-31B-it

MLX 4-bit weights (revision 696d436c, converted from Google's checkpoint)

huggingface.co/mlx-community/gemma-4-31B-it-4bit/tree/696d436c404745a59f30e4939a658162b0a9e57f

Draft model (DFlash, revision eabd6483)

huggingface.co/z-lab/gemma-4-31B-it-DFlash/tree/eabd648301ce28583cc14757912e5e0f84e152e1

Engine

TensorFold 1.2.5, the official Linux release (native CUDA engine, one GB10), MLX 4-bit weights with the DFlash drafter of z-lab, 147,456 tokens of context (set by me, see the changes below), up to 4 streams (--parallel 4, as in the recipe) — the program that loads the model and answers requests · github.com/ashhart/TensorFold/releases/tag/v1.2.5

Recipe (docs/recipes/gemma-4-native.md)

github.com/ashhart/TensorFold/blob/v1.2.5/docs/recipes/gemma-4-native.md by Ash Hart

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888.

   Copy

   tensorfold-native serve gemma-4-31b \
     --port 8888 \
     --parallel 4 \
     --context 147456 \
     --drafter gemma-4-31b-dflash \
     --name gemma-4-31b

   The file names are the ones we used. Run the command in the folder that holds them, or put the full path in front of each name.

   What the parts mean

   • --parallel 4how many requests it answers at the same time
   • --port 8888the network port the server listens on
   • --drafter gemma-4-31b-dflashthe draft model: it guesses the next tokens and the main model checks them; without it the model decodes plain
   • --name gemma-4-31bthe model ID the server advertises to clients

TensorFold 1.2.5 official release archive tensorfold-1.2.5-linux-aarch64.tar.gz (sha256 1a0192b28d6ab6ec2749f3ff25515379dba4895171e781b30f810afc971ae4f8; tensorfold-native sha256 4f7ed2ab...), native CUDA family (Gemma 4), 1 GB10 (sm_121), DFlash drafts, --context 147456, --parallel 4, prompt cache 16 GiB default, text only; tensorfold-native serve $HOME/models/gemma4-31b-1010/gemma-4-31b --port 8888 --parallel 4 --context 147456 --drafter $HOME/models/gemma4-31b-1010/gemma-4-31b-dflash --name gemma-4-31b (Ash's recipe docs/recipes/gemma-4-native.md, lines 15 and 16, plus --context 147456 and --name gemma-4-31b, both named on this page; each folder holds the files of the Hugging Face repository at the revision named above, downloaded with hf download as in lines 10 to 13 of the recipe)