Home / 128 GB

128 GB · 1× DGX Spark (GB10)

Gemma 4 26B-A4B on one DGX Spark: the CapyCTL r3 checkpoint in NVIDIA's recipe (vLLM nightly, MTP 4 tokens)

Weights: NVFP4 (r3 checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 by edurdias, built from google/gemma-4-26B-A4B-it)

1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)

The DGX Spark recipe (NVIDIA/local-ai-inference-recipes), whose arguments are used hereNVIDIA
vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend tritoncommit NVIDIA/local-ai-inference-recipes models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml (recipe_sha256 0a6f5e2d... performance.yaml:2); checkpoint swap as capyctl-recipes bench/runs/vl-r3-official-coding-001/run.json1 GPUmeasured 2026-11-20

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed v2

How fast it runs

Decode
66.3tok/s
Decode, code
70.3tok/s
Decode, prose
62.3tok/s
500 words
≈ 11.3s
Prefill at 8K
6164.5tok/s
First token at 2K
0.2s
MeasurementResult
Decode66.3 tok/s
Words per second44.3 words/s
Characters per second304.6 chars/s
Decode, code prompt70.3 tok/s
Decode, prose prompt62.3 tok/s
Spread of the decode runs (CV)2.5 %
Spread of the code runs (CV)2.5 %
Spread of the prose runs (CV)1.0 %
Decode, window 167.5 tok/s
Prefill at 2K8202.8 tok/s
Prefill at 8K6164.5 tok/s
Prefill at 32K2286.6 tok/s
Time to first token at 2K0.2 s
Time to first token at 8K1.3 s
Time to first token at 32K14.3 s
Draft acceptance rate44.1 %
Mean accepted length2.8 tokens
1 request at once (total)58.1 tok/s
2 at once (total)101.3 tok/s
4 at once (total)166.0 tok/s
8 at once (total)272.2 tok/s
1 request at once (per request)58.1 tok/s
2 at once (per request)51.5 tok/s
4 at once (per request)43.5 tok/s
8 at once (per request)35.2 tok/s
Server slots (8, as set by the recipe: max-num-seqs)8 slots
Long answer, wall time27.8 s

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens4173.8 tok/s3.9 s69.3 tok/s53.5 tok/s
32K tokens2278.3 tok/s14.4 s65.1 tok/s54.4 tok/s
64K tokens1199.2 tok/s54.7 s55.1 tok/s51.1 tok/s
128K tokens626.2 tok/s209.3 s52.5 tok/s44.9 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-20. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 8 slots (from the author's recipe). What ran. NVIDIA's recipe arguments (NVIDIA/local-ai-inference-recipes, models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml) with one change: the checkpoint is capyctl/Gemma-4-26B-A4B-NVFP4 (the r3 build of edurdias) instead of NVIDIA's, as edurdias ran it himself (capyctl-recipes at commit 6ecf465, comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10/bench/runs/vl-r3-official-coding-001/run.json: same flags, kv_cache_dtype auto). It runs in NVIDIA's container (vLLM 0.32.0rc1.dev301) and not in the author's own vLLM 0.32.0; Google's MTP drafter with 4 speculative tokens, moe_backend triton, window 262,144 tokens, 8 slots.

The headline 66.3 tok/s is the mean of the code prompt (70.3) and the prose prompt (62.3), medians of two runs each (spread (CV) 2.5 % and 1.0 %). The code prompt has 1,379 and 1,382 tokens and the prose prompt 1,350 and 1,351 tokens; every reply is 512 tokens long and ends at the limit. About 11.3 s for 500 words (44.29 words per second). Accepted drafts: 44.1 %.

Long prompts. Prefill at 2K, 8K and 32K tokens: 8202.8, 6164.5 and 2286.6 tok/s. Context steps (decode on the code prompt after a prompt of that size): 16K 69.3 tok/s (prefill 4173.8), 32K 65.1 (prefill 2278.3), 64K 55.1 (prefill 1199.2) and the 128K step 52.5 tok/s (prefill 626.2 tok/s, first token after 209.3 s). The 128K step is measured at 131,072 tokens (the server window is 262,144 tokens, so a prompt of exactly 131,072 tokens fits).

Requests at once. The server has 8 slots (as set by the recipe), so 1, 2, 4 and 8 requests run in parallel: the total grows to 58.1, 101.3, 166.0 and 272.2 tok/s with 1, 2, 4 and 8 requests and the speed of each request falls to 58.1, 51.5, 43.5 and 35.2 tok/s.

Sampling. The speed test sent the Gemma 4 card's sampling (temperature 1.0, top_p 0.95, top_k 64) with thinking off in the request body, seed 42 and cache_prompt false; two runs per prompt of 512 tokens, medians. Cache check: the check of the test tool is marked not ok: 0 of the 61 requests report cached tokens (the details are in the raw speed result).

Foreign load. The load guard repeated 1 of 38 steps (1 repeats); details in the guard section of the speed result.

The author's published numbers for this setup, measured with his own benchmark: r3 98.79 tok/s (plus or minus 7.85) against 81.46 (plus or minus 7.63) for NVIDIA's checkpoint (capyctl-recipes at commit 6ecf465, comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10, README lines 354 to 358). We did not run that benchmark on this page; the table shows the standard test.

Note on the engine line in the raw speed result: the tool's version probe ran inside the container and did not work there (the line shows an error text instead of a version). The engine is vLLM 0.32.0rc1.dev301, the version the container reports in its own start log (the container image is eugr/spark-vllm:nightly-20261016); the start log is not published.

Only Speed v2 was run on this setup. The author's own numbers, where listed above, are the author's published numbers measured with another method; they are not a measurement of ours and not directly comparable with the table.

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run gemma4-r3-nvrecipe-mtp4-2011-speed-r0)
python3 speed2.py --base-url http://127.0.0.1:40557/v1 --label gemma4-r3-nvrecipe-mtp4-2011-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/speed2 --contract $RUN/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/gemma4-r3-nvrecipe-mtp4-2011-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/capyctl--Gemma-4-26B-A4B-NVFP4@241dbe0757d0fca815530e9216de6affb43565ff/tokenizer.json --tokenizer-python $HOME/gemma4-capyctl-1911/vllm-0.32.0-venv/bin/python --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64, "min_p": 0.0} --sampling-source google/gemma-4-26B-A4B-it generation_config.json = capyctl/Gemma-4-26B-A4B-NVFP4@241dbe07 generation_config.json (temperature 1.0, top_k 64, top_p 0.95); same Gemma 4 Best Practices as card mac-gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-m5ultra-1010. No presence penalty. Seed 42 for speed. Thinking off through request body enable_thinking=false (chat template default is also off). --model capyctl/Gemma-4-26B-A4B-NVFP4 --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 8 --server-slots-source NVIDIA/local-ai-inference-recipes models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml:80 (--max-num-seqs 8); concurrency: 8 (recipe.yaml:42) --server-ctx 262144 --server-ctx-source card GB10 box 2-gemma4-26b-a4b-capyctl-r3-nvfp4-vllm0320-mtp4-nvidia-recipe-1x-2011 field ctx
Quality v2

Quality

Not run on this setup: no Quality v2 test, so the page shows no quality number.

Credits

Also made this possible

  • The r3 checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 and the comparison run that this setup follows (capyctl-recipes at commit 6ecf465)Eduardo Dias (edurdias)GitHub
  • Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistantGoogleHugging Face
  • Engine (vLLM)vLLM projectGitHub
Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)
Model
capyctl/Gemma-4-26B-A4B-NVFP4 @241dbe07
Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistant
huggingface.co/google/gemma-4-26B-A4B-it
Engine
NVIDIA's DGX Spark recipe with the r3 checkpoint swapped in, as edurdias ran it; Speed v2 only — the program that loads the model and answers requests · github.com/vllm-project/vllmBuild and settings we used: vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend triton

Steps

  1. Get the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory.

    bash <start script of the run>
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend triton; bash <start script of the run> (starts the container eugr/spark-vllm:nightly-20261016 with the arguments of NVIDIA's recipe models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml, with the checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 r3 instead of NVIDIA's: vLLM 0.32.0rc1.dev301, MTP drafter with 4 speculative tokens, moe_backend triton, max-model-len 262144, kv-cache-dtype auto)

Context and sampling: 262,144 tokens (the recipe's --max-model-len), KV cache type auto (as in the author's run; the checkpoint selects FP8 E4M3), sampling of the Gemma 4 card (temperature 1.0, top_p 0.95, top_k 64), seed 42, thinking off via the request body, 8 server slots.. Hardware: 1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages). All raw files of this setup: raw/v2/gemma4-26b-a4b-1spark-capyctl-r3-nvfp4-vllm-mtp4-nvidia-recipe-r3-checkpoint.