Gemma 4 26B-A4B on one DGX Spark: the CapyCTL r3 checkpoint in NVIDIA's recipe (vLLM nightly, MTP 4 tokens)
Weights: NVFP4 (r3 checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 by edurdias, built from google/gemma-4-26B-A4B-it)
1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)
Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.
How fast it runs
| Measurement | Result |
|---|---|
| Decode | 66.3 tok/s |
| Words per second | 44.3 words/s |
| Characters per second | 304.6 chars/s |
| Decode, code prompt | 70.3 tok/s |
| Decode, prose prompt | 62.3 tok/s |
| Spread of the decode runs (CV) | 2.5 % |
| Spread of the code runs (CV) | 2.5 % |
| Spread of the prose runs (CV) | 1.0 % |
| Decode, window 1 | 67.5 tok/s |
| Prefill at 2K | 8202.8 tok/s |
| Prefill at 8K | 6164.5 tok/s |
| Prefill at 32K | 2286.6 tok/s |
| Time to first token at 2K | 0.2 s |
| Time to first token at 8K | 1.3 s |
| Time to first token at 32K | 14.3 s |
| Draft acceptance rate | 44.1 % |
| Mean accepted length | 2.8 tokens |
| 1 request at once (total) | 58.1 tok/s |
| 2 at once (total) | 101.3 tok/s |
| 4 at once (total) | 166.0 tok/s |
| 8 at once (total) | 272.2 tok/s |
| 1 request at once (per request) | 58.1 tok/s |
| 2 at once (per request) | 51.5 tok/s |
| 4 at once (per request) | 43.5 tok/s |
| 8 at once (per request) | 35.2 tok/s |
| Server slots (8, as set by the recipe: max-num-seqs) | 8 slots |
| Long answer, wall time | 27.8 s |
Long prompts
| Prompt length | Prefill | Time to first token | Decode, code prompt | Decode, prose prompt |
|---|---|---|---|---|
| 16K tokens | 4173.8 tok/s | 3.9 s | 69.3 tok/s | 53.5 tok/s |
| 32K tokens | 2278.3 tok/s | 14.4 s | 65.1 tok/s | 54.4 tok/s |
| 64K tokens | 1199.2 tok/s | 54.7 s | 55.1 tok/s | 51.1 tok/s |
| 128K tokens | 626.2 tok/s | 209.3 s | 52.5 tok/s | 44.9 tok/s |
Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-20. Raw files: speed result · speed contract · log · long answer.
Requests at once: the server has 8 slots (from the author's recipe). What ran. NVIDIA's recipe arguments (NVIDIA/local-ai-inference-recipes, models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml) with one change: the checkpoint is capyctl/Gemma-4-26B-A4B-NVFP4 (the r3 build of edurdias) instead of NVIDIA's, as edurdias ran it himself (capyctl-recipes at commit 6ecf465, comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10/bench/runs/vl-r3-official-coding-001/run.json: same flags, kv_cache_dtype auto). It runs in NVIDIA's container (vLLM 0.32.0rc1.dev301) and not in the author's own vLLM 0.32.0; Google's MTP drafter with 4 speculative tokens, moe_backend triton, window 262,144 tokens, 8 slots.
The headline 66.3 tok/s is the mean of the code prompt (70.3) and the prose prompt (62.3), medians of two runs each (spread (CV) 2.5 % and 1.0 %). The code prompt has 1,379 and 1,382 tokens and the prose prompt 1,350 and 1,351 tokens; every reply is 512 tokens long and ends at the limit. About 11.3 s for 500 words (44.29 words per second). Accepted drafts: 44.1 %.
Long prompts. Prefill at 2K, 8K and 32K tokens: 8202.8, 6164.5 and 2286.6 tok/s. Context steps (decode on the code prompt after a prompt of that size): 16K 69.3 tok/s (prefill 4173.8), 32K 65.1 (prefill 2278.3), 64K 55.1 (prefill 1199.2) and the 128K step 52.5 tok/s (prefill 626.2 tok/s, first token after 209.3 s). The 128K step is measured at 131,072 tokens (the server window is 262,144 tokens, so a prompt of exactly 131,072 tokens fits).
Requests at once. The server has 8 slots (as set by the recipe), so 1, 2, 4 and 8 requests run in parallel: the total grows to 58.1, 101.3, 166.0 and 272.2 tok/s with 1, 2, 4 and 8 requests and the speed of each request falls to 58.1, 51.5, 43.5 and 35.2 tok/s.
Sampling. The speed test sent the Gemma 4 card's sampling (temperature 1.0, top_p 0.95, top_k 64) with thinking off in the request body, seed 42 and cache_prompt false; two runs per prompt of 512 tokens, medians. Cache check: the check of the test tool is marked not ok: 0 of the 61 requests report cached tokens (the details are in the raw speed result).
Foreign load. The load guard repeated 1 of 38 steps (1 repeats); details in the guard section of the speed result.
The author's published numbers for this setup, measured with his own benchmark: r3 98.79 tok/s (plus or minus 7.85) against 81.46 (plus or minus 7.63) for NVIDIA's checkpoint (capyctl-recipes at commit 6ecf465, comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10, README lines 354 to 358). We did not run that benchmark on this page; the table shows the standard test.
Note on the engine line in the raw speed result: the tool's version probe ran inside the container and did not work there (the line shows an error text instead of a version). The engine is vLLM 0.32.0rc1.dev301, the version the container reports in its own start log (the container image is eugr/spark-vllm:nightly-20261016); the start log is not published.
Only Speed v2 was run on this setup. The author's own numbers, where listed above, are the author's published numbers measured with another method; they are not a measurement of ours and not directly comparable with the table.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 speed2.py --base-url http://127.0.0.1:40557/v1 --label gemma4-r3-nvrecipe-mtp4-2011-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/speed2 --contract $RUN/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/capyctl--Gemma-4-26B-A4B-NVFP4@241dbe0757d0fca815530e9216de6affb43565ff/tokenizer.json --tokenizer-python $HOME/gemma4-capyctl-1911/vllm-0.32.0-venv/bin/python --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64, "min_p": 0.0} --sampling-source google/gemma-4-26B-A4B-it generation_config.json = capyctl/Gemma-4-26B-A4B-NVFP4@241dbe07 generation_config.json (temperature 1.0, top_k 64, top_p 0.95); same Gemma 4 Best Practices as card mac-gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-m5ultra-1010. No presence penalty. Seed 42 for speed. Thinking off through request body enable_thinking=false (chat template default is also off). --model capyctl/Gemma-4-26B-A4B-NVFP4 --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 8 --server-slots-source NVIDIA/local-ai-inference-recipes models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml:80 (--max-num-seqs 8); concurrency: 8 (recipe.yaml:42) --server-ctx 262144 --server-ctx-source card GB10 box 2-gemma4-26b-a4b-capyctl-r3-nvfp4-vllm0320-mtp4-nvidia-recipe-1x-2011 field ctxQuality
Not run on this setup: no Quality v2 test, so the page shows no quality number.
Also made this possible
- The r3 checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 and the comparison run that this setup follows (capyctl-recipes at commit 6ecf465)EEduardo Dias (edurdias)GitHub
- Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistantGGoogleHugging Face
- Engine (vLLM)VvLLM projectGitHub
Run it yourself
What you need and how to start it, step by step. The results below come from exactly this setup.
What you need
- Hardware
- 1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)
- Model
- capyctl/Gemma-4-26B-A4B-NVFP4 @241dbe07
- Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistant
- huggingface.co/google/gemma-4-26B-A4B-it
- Engine
- NVIDIA's DGX Spark recipe with the r3 checkpoint swapped in, as edurdias ran it; Speed v2 only — the program that loads the model and answers requests · github.com/vllm-project/vllmBuild and settings we used: vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend triton
Steps
Get the engine and the model files from the list above.
Start the server with this command. It loads the model into memory.
bash <start script of the run>
Start text for this setupfor reproducing↓
You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.
vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend triton; bash <start script of the run> (starts the container eugr/spark-vllm:nightly-20261016 with the arguments of NVIDIA's recipe models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml, with the checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 r3 instead of NVIDIA's: vLLM 0.32.0rc1.dev301, MTP drafter with 4 speculative tokens, moe_backend triton, max-model-len 262144, kv-cache-dtype auto)Context and sampling: 262,144 tokens (the recipe's --max-model-len), KV cache type auto (as in the author's run; the checkpoint selects FP8 E4M3), sampling of the Gemma 4 card (temperature 1.0, top_p 0.95, top_k 64), seed 42, thinking off via the request body, 8 server slots.. Hardware: 1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages). All raw files of this setup: raw/v2/gemma4-26b-a4b-1spark-capyctl-r3-nvfp4-vllm-mtp4-nvidia-recipe-r3-checkpoint.