Home / 128 GB

128 GB · 1× DGX Spark (GB10)

Gemma 4 26B-A4B on one DGX Spark: NVIDIA's own recipe (NVFP4, vLLM nightly, MTP 4 tokens)

Weights: NVFP4 (NVIDIA's checkpoint nvidia/Gemma-4-26B-A4B-NVFP4)

1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)

NVFP4 checkpoint nvidia/Gemma-4-26B-A4B-NVFP4 and the DGX Spark recipe (NVIDIA/local-ai-inference-recipes)NVIDIA
vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend tritoncommit NVIDIA/local-ai-inference-recipes models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml (recipe_sha256 0a6f5e2d... performance.yaml:2)1 GPUmeasured 2026-11-20

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed v2

How fast it runs

Decode
53.8tok/s
Decode, code
59.5tok/s
Decode, prose
48.2tok/s
500 words
≈ 13.4s
Prefill at 8K
5477.2tok/s
First token at 2K
0.3s
MeasurementResult
Decode53.8 tok/s
Words per second37.3 words/s
Characters per second252.8 chars/s
Decode, code prompt59.5 tok/s
Decode, prose prompt48.2 tok/s
Spread of the decode runs (CV)6.5 %
Spread of the code runs (CV)6.5 %
Spread of the prose runs (CV)0.6 %
Decode, window 153.5 tok/s
Prefill at 2K6921.3 tok/s
Prefill at 8K5477.2 tok/s
Prefill at 32K2186.9 tok/s
Time to first token at 2K0.3 s
Time to first token at 8K1.5 s
Time to first token at 32K15.0 s
Draft acceptance rate43.3 %
Mean accepted length2.7 tokens
1 request at once (total)49.2 tok/s
2 at once (total)87.4 tok/s
4 at once (total)142.6 tok/s
8 at once (total)232.3 tok/s
1 request at once (per request)49.2 tok/s
2 at once (per request)43.8 tok/s
4 at once (per request)37.3 tok/s
8 at once (per request)29.9 tok/s
Server slots (8, as set by the recipe: max-num-seqs)8 slots
Long answer, wall time32.5 s

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens3868.7 tok/s4.2 s58.7 tok/s47.0 tok/s
32K tokens2172.0 tok/s15.1 s53.1 tok/s44.2 tok/s
64K tokens1168.5 tok/s56.1 s53.2 tok/s42.0 tok/s
128K tokens616.5 tok/s212.6 s44.4 tok/s39.2 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-20. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 8 slots (from the author's recipe). What ran. NVIDIA's recipe for the DGX Spark (NVIDIA/local-ai-inference-recipes, models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml, lines 53 to 87; the repository belongs to the organisation NVIDIA), run as written in the container image eugr/spark-vllm:nightly-20261016 (vLLM 0.32.0rc1.dev301): NVIDIA's checkpoint nvidia/Gemma-4-26B-A4B-NVFP4, Google's MTP drafter with 4 speculative tokens, moe_backend triton, window 262,144 tokens, gpu-memory-utilization 0.8, 8 slots, prefix caching on. The recipe sets no KV cache type, so vLLM's default applies.

The headline 53.8 tok/s is the mean of the code prompt (59.5) and the prose prompt (48.2), medians of two runs each (spread (CV) 6.5 % and 0.6 %). The code prompt has 1,380 and 1,382 tokens and the prose prompt 1,345 and 1,350 tokens; every reply is 512 tokens long and ends at the limit. About 13.4 s for 500 words (37.31 words per second). Accepted drafts: 43.3 %.

Long prompts. Prefill at 2K, 8K and 32K tokens: 6921.3, 5477.2 and 2186.9 tok/s. Context steps (decode on the code prompt after a prompt of that size): 16K 58.7 tok/s (prefill 3868.7), 32K 53.1 (prefill 2172.0), 64K 53.2 (prefill 1168.5) and the 128K step 44.4 tok/s (prefill 616.5 tok/s, first token after 212.6 s). The 128K step is measured at 131,072 tokens (the server window is 262,144 tokens, so a prompt of exactly 131,072 tokens fits).

Requests at once. The server has 8 slots (as set by the recipe), so 1, 2, 4 and 8 requests run in parallel: the total grows to 49.2, 87.4, 142.6 and 232.3 tok/s with 1, 2, 4 and 8 requests and the speed of each request falls to 49.2, 43.8, 37.3 and 29.9 tok/s.

Sampling. The speed test sent the Gemma 4 card's sampling (temperature 1.0, top_p 0.95, top_k 64) with thinking off in the request body, seed 42 and cache_prompt false; two runs per prompt of 512 tokens, medians. Cache check: the check of the test tool is marked not ok: 0 of the 61 requests report cached tokens (the details are in the raw speed result).

Foreign load. The load guard repeated 0 of 38 steps (0 repeats).

NVIDIA's own published number for this recipe: 78.10 tok/s decode with one stream (performance.yaml line 57), measured with NVIDIA's benchmark (aiperf with the SPEED-Bench coding set, one stream, 10 requests, 3 warm-up requests). The standard test of this page is another benchmark, so the table does not show that number. Not the same configuration as the CapyCTL page: another vLLM build (0.32.0rc1.dev301 here, 0.32.0 there), 4 drafted tokens instead of 3, a window of 262,144 tokens instead of 131,072 and the default KV cache type instead of FP8; the two pages are not a like-for-like comparison.

Note on the engine line in the raw speed result: the tool's version probe ran inside the container and did not work there (the line shows an error text instead of a version). The engine is vLLM 0.32.0rc1.dev301, the version the container reports in its own start log (the container image is eugr/spark-vllm:nightly-20261016); the start log is not published.

Only Speed v2 was run on this setup. The author's own numbers, where listed above, are the author's published numbers measured with another method; they are not a measurement of ours and not directly comparable with the table.

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run gemma4-nvidia-vllm-mtp4-2011b-speed-r0)
python3 speed2.py --base-url http://127.0.0.1:39847/v1 --label gemma4-nvidia-vllm-mtp4-2011b-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/gemma4-nvidia-vllm-mtp4-2011b-speed-r0/gemma4-nvidia-vllm-mtp4-2011b-speed-r0/speed2 --contract $RUN/gemma4-nvidia-vllm-mtp4-2011b-speed-r0/gemma4-nvidia-vllm-mtp4-2011b-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/nvidia--Gemma-4-26B-A4B-NVFP4@a19cfe00be84568a6867111c9a68c9c44fdcffe6/tokenizer.json --tokenizer-python $HOME/gemma4-capyctl-1911/vllm-0.32.0-venv/bin/python --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64, "min_p": 0.0} --sampling-source google/gemma-4-26B-A4B-it generation_config.json = capyctl/Gemma-4-26B-A4B-NVFP4@241dbe07 generation_config.json (temperature 1.0, top_k 64, top_p 0.95); same Gemma 4 Best Practices as card mac-gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-m5ultra-1010. No presence penalty. Seed 42 for speed. Thinking off through request body enable_thinking=false (chat template default is also off). --model nvidia/Gemma-4-26B-A4B-NVFP4 --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 8 --server-slots-source NVIDIA/local-ai-inference-recipes models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml:80 (--max-num-seqs 8); concurrency: 8 (recipe.yaml:42) --server-ctx 262144 --server-ctx-source card GB10 box 2-gemma4-26b-a4b-nvidia-nvfp4-vllm0320-mtp4-nvidia-recipe-1x-2011 field ctx
Quality v2

Quality

Not run on this setup: no Quality v2 test, so the page shows no quality number.

Credits

Also made this possible

  • Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistantGoogleHugging Face
  • Engine (vLLM)vLLM projectGitHub
Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)
Model
nvidia/Gemma-4-26B-A4B-NVFP4 @a19cfe00
Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistant
huggingface.co/google/gemma-4-26B-A4B-it
Engine
NVIDIA's DGX Spark recipe as published (NVIDIA/local-ai-inference-recipes); Speed v2 only — the program that loads the model and answers requests · github.com/vllm-project/vllmBuild and settings we used: vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend triton

Steps

  1. Get the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory.

    bash <start script of the run>
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
vLLM 0.32.0rc1.dev301+g1ac942555 (container eugr/spark-vllm:nightly-20261016), Google MTP drafter, 4 speculative tokens, moe_backend triton; bash <start script of the run> (starts the container eugr/spark-vllm:nightly-20261016 with NVIDIA's recipe models/google/gemma-4-26b-a4b/dgx-spark/linux/vllm/128gb/recipe.yaml: vLLM 0.32.0rc1.dev301, checkpoint nvidia/Gemma-4-26B-A4B-NVFP4, MTP drafter with 4 speculative tokens, moe_backend triton, max-model-len 262144, gpu-memory-utilization 0.8, max-num-seqs 8, prefix caching)

Context and sampling: 262,144 tokens (the recipe's --max-model-len), KV cache type not set by the recipe (vLLM default), sampling of the Gemma 4 card (temperature 1.0, top_p 0.95, top_k 64), seed 42, thinking off via the request body, 8 server slots (the recipe's max-num-seqs).. Hardware: 1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages). All raw files of this setup: raw/v2/gemma4-26b-a4b-1spark-nvidia-nvfp4-vllm-mtp4-nvidia-recipe.