Home / 128 GB

128 GB · 1× DGX Spark (GB10)

Gemma 4 26B-A4B on one DGX Spark: NVIDIA's NVFP4 checkpoint on SGLang, no drafter

Weights: NVFP4 (NVIDIA's checkpoint nvidia/Gemma-4-26B-A4B-NVFP4)

1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)

Comparison entry sg-nvidia-none of the recipes (capyctl-recipes at commit 6ecf465) and CapyCTLEduardo Dias (edurdias)
SGLang 0.5.22 through CapyCTL 0.1.12 standalone, no drafter, flashinfer_cutlass MoE, lmcache offcommit capyctl-recipes @6ecf465 comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10/matrix.py:14-65 (key sg-nvidia-none)1 GPUmeasured 2026-11-20

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed v2

How fast it runs

Decode
29.0tok/s
Decode, code
29.0tok/s
Decode, prose
28.9tok/s
500 words
≈ 26.3s
Prefill at 8K
5015.8tok/s
First token at 2K
0.4s
MeasurementResult
Decode29.0 tok/s
Words per second19.0 words/s
Characters per second134.6 chars/s
Decode, code prompt29.0 tok/s
Decode, prose prompt28.9 tok/s
Spread of the decode runs (CV)0.4 %
Spread of the code runs (CV)0.4 %
Spread of the prose runs (CV)0.4 %
Decode, window 128.9 tok/s
Prefill at 2K5789.0 tok/s
Prefill at 8K5015.8 tok/s
Prefill at 32K3579.5 tok/s
Time to first token at 2K0.4 s
Time to first token at 8K1.6 s
Time to first token at 32K9.2 s
1 request at once (total)28.3 tok/s
2 at once (total)60.1 tok/s
4 at once (total)102.0 tok/s
8 at once (total)167.1 tok/s
1 request at once (per request)28.3 tok/s
2 at once (per request)30.1 tok/s
4 at once (per request)25.5 tok/s
8 at once (per request)20.9 tok/s
Server slots (8, entered in our run card; the recipe sets no value for it)8 slots
Long answer, wall time83.3 s

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens4587.0 tok/s3.6 s27.9 tok/s27.9 tok/s
32K tokens3675.9 tok/s8.9 s26.8 tok/s26.9 tok/s
64K tokens2634.1 tok/s24.9 s25.0 tok/s25.1 tok/s
128K tokens1689.0 tok/s77.0 s21.9 tok/s21.9 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-20. Raw files: speed result · speed contract · log · long answer.

What ran. The comparison entry sg-nvidia-none of edurdias/capyctl-recipes at commit 6ecf465 (comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10, matrix.py lines 14 to 65), started with CapyCTL 0.1.12: SGLang 0.5.22 and NVIDIA's checkpoint nvidia/Gemma-4-26B-A4B-NVFP4, without a drafter (no speculative decoding), MoE backend flashinfer_cutlass, FP8 (E4M3) KV cache, window 131,072 tokens. This is the base that edurdias compares his r3 build against; it is not NVIDIA's own recipe.

The headline 29.0 tok/s is the mean of the code prompt (29.0) and the prose prompt (28.9), medians of two runs each (spread (CV) 0.4 % and 0.4 %). The code prompt has 1,377 and 1,378 tokens and the prose prompt 1,349 and 1,352 tokens; every reply is 512 tokens long and ends at the limit. About 26.3 s for 500 words (19.05 words per second). Accepted drafts: not applicable (no drafter on this setup).

Long prompts. Prefill at 2K, 8K and 32K tokens: 5789.0, 5015.8 and 3579.5 tok/s. Context steps (decode on the code prompt after a prompt of that size): 16K 27.9 tok/s (prefill 4587.0), 32K 26.8 (prefill 3675.9), 64K 25.0 (prefill 2634.1) and the 128K step 21.9 tok/s (prefill 1689.0 tok/s, first token after 77.0 s). The 128K step is measured at 130,048 tokens (the server window of 131,072 tokens minus 1,024, because a prompt of exactly 131,072 tokens plus the reply would not fit).

Requests at once. The run card enters 8 slots (the recipe sets no value), so 1, 2, 4 and 8 requests run in parallel: the total grows to 28.3, 60.1, 102.0 and 167.1 tok/s with 1, 2, 4 and 8 requests and the speed of each request falls to 28.3, 30.1, 25.5 and 20.9 tok/s.

Sampling. The speed test sent the Gemma 4 card's sampling (temperature 1.0, top_p 0.95, top_k 64) with thinking off in the request body, seed 42 and cache_prompt false; two runs per prompt of 512 tokens, medians. Cache check: ok, no request reported cached tokens.

Foreign load. The load guard repeated 4 of 38 steps (5 repeats); details in the guard section of the speed result.

The author's published number for this entry, measured with his own benchmark: 29.47 tok/s with one stream and 167.67 tok/s with eight streams (capyctl-recipes at commit 6ecf465, comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10, README line 181). Without a drafter the decode is slower than with the MTP drafter on short prompts, but it falls much less with long prompts. Its prefill is the lowest of the four Gemma setups at 2K and 8K tokens and the highest at 32K tokens (3579.5 tok/s).

Only Speed v2 was run on this setup. The author's own numbers, where listed above, are the author's published numbers measured with another method; they are not a measurement of ours and not directly comparable with the table.

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run gemma4-nvidia-sg-1911b-speed-r0)
python3 speed2.py --base-url http://127.0.0.1:39947/v1 --label gemma4-nvidia-sg-1911b-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/gemma4-nvidia-sg-1911b-speed-r0/gemma4-nvidia-sg-1911b-speed-r0/speed2 --contract $RUN/gemma4-nvidia-sg-1911b-speed-r0/gemma4-nvidia-sg-1911b-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/nvidia--Gemma-4-26B-A4B-NVFP4@a19cfe00be84568a6867111c9a68c9c44fdcffe6/tokenizer.json --tokenizer-python $HOME/gemma4-capyctl-1911/vllm-0.32.0-venv/bin/python --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64, "min_p": 0.0} --sampling-source google/gemma-4-26B-A4B-it generation_config.json = capyctl/Gemma-4-26B-A4B-NVFP4@241dbe07 generation_config.json (temperature 1.0, top_k 64, top_p 0.95); same Gemma 4 Best Practices as card mac-gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-m5ultra-1010. No presence penalty. Seed 42 for speed. Thinking off through request body enable_thinking=false (chat template default is also off). --model g4-sg-nvidia-none --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 8 --server-slots-source capyctl-recipes@6ecf465 comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10/matrix.py:28-65 sets no max_running_requests; SGLang 0.5.22 engine log max_running_requests=2343 (raw/author-nvsg-0143/engine-logs); CapyCTL router keeps at most 32 requests in flight (edurdias/capyctl issue 240) --server-ctx 131072 --server-ctx-source card GB10 box 2-gemma4-26b-a4b-nvidia-nvfp4-sglang0522-nodraft-capyctl0112-1x-1911 field ctx
Quality v2

Quality

Not run on this setup: no Quality v2 test, so the page shows no quality number.

Credits

Also made this possible

  • NVFP4 checkpoint nvidia/Gemma-4-26B-A4B-NVFP4 (NVIDIA)NVIDIAGitHub
  • Gemma 4 26B-A4B (base model)GoogleHugging Face
  • Engine (SGLang 0.5.22)SGLang projectGitHub
Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)
Model
nvidia/Gemma-4-26B-A4B-NVFP4 @a19cfe00
Gemma 4 26B-A4B (base model)
huggingface.co/google/gemma-4-26B-A4B-it
Engine
The comparison base of edurdias' recipes (capyctl-recipes at commit 6ecf465, entry sg-nvidia-none); Speed v2 only — the program that loads the model and answers requests · github.com/sgl-project/sglangBuild and settings we used: SGLang 0.5.22 through CapyCTL 0.1.12 standalone, no drafter, flashinfer_cutlass MoE, lmcache off

Steps

  1. Get the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory.

    bash <start script of the run>
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
SGLang 0.5.22 through CapyCTL 0.1.12 standalone, no drafter, flashinfer_cutlass MoE, lmcache off; bash <start script of the run> (starts CapyCTL 0.1.12 with the comparison entry sg-nvidia-none of capyctl-recipes at commit 6ecf465: SGLang 0.5.22, checkpoint nvidia/Gemma-4-26B-A4B-NVFP4, no drafter, flashinfer_cutlass MoE backend, FP8 KV cache, context 131072)

Context and sampling: 131,072 tokens (the recipe's context_length), FP8 (E4M3) KV cache, the recipe's parsers, sampling of the Gemma 4 card (temperature 1.0, top_p 0.95, top_k 64), seed 42, thinking off via the request body, no drafter, 8 server slots (entered in our run card; the recipe sets no value for it).. Hardware: 1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages). All raw files of this setup: raw/v2/gemma4-26b-a4b-1spark-nvidia-nvfp4-sglang-no-drafter-capyctl-recipe-edurdias.