Home / 128 GB

128 GB · 1× DGX Spark (GB10)

Gemma 4 26B-A4B on one DGX Spark: CapyCTL r3 (NVFP4, vLLM, MTP 3 tokens)

Weights: NVFP4 (r3 checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 by edurdias, built from google/gemma-4-26B-A4B-it)

1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)

CapyCTL, the recipes (capyctl-recipes at commit 6ecf465) and the r3 checkpointEduardo Dias (edurdias)
vLLM 0.32.0 (torch 2.13.0+cu130) through CapyCTL 0.1.12 standalone, Google MTP drafter depth 3 (google/gemma-4-26B-A4B-it-assistant @6e5aaaf4), lmcache offcommit recipe capyctl-recipes @6ecf465 (vllm/gemma-4-26b-a4b-capyctl-nvfp4-gb10/deployment.yaml)1 GPUmeasured 2026-11-20

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed v2

How fast it runs

Decode
67.3tok/s
Decode, code
71.1tok/s
Decode, prose
63.4tok/s
500 words
≈ 11.0s
Prefill at 8K
5710.2tok/s
First token at 2K
0.3s
MeasurementResult
Decode67.3 tok/s
Words per second45.3 words/s
Characters per second309.9 chars/s
Decode, code prompt71.1 tok/s
Decode, prose prompt63.4 tok/s
Spread of the decode runs (CV)1.9 %
Spread of the code runs (CV)1.9 %
Spread of the prose runs (CV)1.0 %
Decode, window 169.9 tok/s
Prefill at 2K7922.3 tok/s
Prefill at 8K5710.2 tok/s
Prefill at 32K2214.2 tok/s
Time to first token at 2K0.3 s
Time to first token at 8K1.4 s
Time to first token at 32K14.8 s
1 request at once (total)61.3 tok/s
2 at once (total)103.5 tok/s
4 at once (total)172.0 tok/s
8 at once (total)277.8 tok/s
1 request at once (per request)61.3 tok/s
2 at once (per request)52.2 tok/s
4 at once (per request)44.4 tok/s
8 at once (per request)36.0 tok/s
Server slots (8, entered in our run card; the recipe sets no value for it)8 slots
Long answer, wall time27.2 s

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens4273.8 tok/s3.8 s49.5 tok/s39.8 tok/s
32K tokens2294.2 tok/s14.3 s37.2 tok/s31.3 tok/s
64K tokens1197.3 tok/s54.7 s23.8 tok/s20.3 tok/s
128K tokens627.8 tok/s207.2 s13.6 tok/s12.2 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-20. Raw files: speed result · speed contract · log · long answer.

What ran. The recipe vllm/gemma-4-26b-a4b-capyctl-nvfp4-gb10 of edurdias/capyctl-recipes at commit 6ecf465 (author and repository owner: Eduardo Dias, edurdias), started with CapyCTL 0.1.12: vLLM 0.32.0, the r3 checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 (built by the author from google/gemma-4-26B-A4B-it) and Google's MTP drafter google/gemma-4-26B-A4B-it-assistant with 3 speculative tokens. Window 131,072 tokens and FP8 (E4M3) KV cache, as the recipe sets them (deployment.yaml lines 5 to 28). Nothing was lowered.

The headline 67.3 tok/s is the mean of the code prompt (71.1) and the prose prompt (63.4), medians of two runs each (spread (CV) 1.9 % and 1.0 %). The code prompt has 1,379 and 1,381 tokens and the prose prompt 1,350 and 1,352 tokens; every reply is 512 tokens long and ends at the limit. About 11.0 s for 500 words (45.27 words per second). Accepted drafts: not reported by the engine.

Long prompts. Prefill at 2K, 8K and 32K tokens: 7922.3, 5710.2 and 2214.2 tok/s. Context steps (decode on the code prompt after a prompt of that size): 16K 49.5 tok/s (prefill 4273.8), 32K 37.2 (prefill 2294.2), 64K 23.8 (prefill 1197.3) and the 128K step 13.6 tok/s (prefill 627.8 tok/s, first token after 207.2 s). The 128K step is measured at 130,048 tokens (the server window of 131,072 tokens minus 1,024, because a prompt of exactly 131,072 tokens plus the reply would not fit).

Requests at once. The run card enters 8 slots (the recipe sets no value), so 1, 2, 4 and 8 requests run in parallel: the total grows to 61.3, 103.5, 172.0 and 277.8 tok/s with 1, 2, 4 and 8 requests and the speed of each request falls to 61.3, 52.2, 44.4 and 36.0 tok/s.

Sampling. The speed test sent the Gemma 4 card's sampling (temperature 1.0, top_p 0.95, top_k 64) with thinking off in the request body, seed 42 and cache_prompt false; two runs per prompt of 512 tokens, medians. Cache check: the check of the test tool is marked not ok: 0 of the 61 requests report cached tokens (the details are in the raw speed result).

Foreign load. The load guard repeated 1 of 38 steps (1 repeats); details in the guard section of the speed result.

The author's published number for this recipe, measured with his own benchmark (his measure.py, 8 prompts, temperature 0, 512 tokens, 3 rounds): 78.9 tok/s with one stream and 302.3 tok/s with eight streams (capyctl-recipes at commit 6ecf465, comparisons/gemma-4-26b-a4b-nvfp4-builds-gb10, README line 294; 78.86 and 302.29 in his measure.json of the run vl-r3-mtp-3-speed-001). This page's standard test is another benchmark (two prompts, sampled at temperature 1.0), so the numbers on this page are not that number. With long prompts the decode of this setup falls a lot: the author shows the same effect himself in the recipe's README (70.00 tok/s at 500 tokens of input, 29.45 at 32,000 tokens, README of the recipe folder, lines 172 to 180).

Only Speed v2 was run on this setup. The author's own numbers, where listed above, are the author's published numbers measured with another method; they are not a measurement of ours and not directly comparable with the table.

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run gemma4-capyctl-r3-1911b-speed-r0)
python3 speed2.py --base-url http://127.0.0.1:44703/v1 --label gemma4-capyctl-r3-1911b-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/gemma4-capyctl-r3-1911b-speed-r0/gemma4-capyctl-r3-1911b-speed-r0/speed2 --contract $RUN/gemma4-capyctl-r3-1911b-speed-r0/gemma4-capyctl-r3-1911b-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/capyctl--Gemma-4-26B-A4B-NVFP4@241dbe0757d0fca815530e9216de6affb43565ff/tokenizer.json --tokenizer-python $HOME/gemma4-capyctl-1911/vllm-0.32.0-venv/bin/python --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64, "min_p": 0.0} --sampling-source google/gemma-4-26B-A4B-it generation_config.json = capyctl/Gemma-4-26B-A4B-NVFP4@241dbe07 generation_config.json (temperature 1.0, top_k 64, top_p 0.95); same Gemma 4 Best Practices as card mac-gemma-4-31b-mlx4bit-tf-1.2.5-official-dflash-m5ultra-1010. No presence penalty. Seed 42 for speed. Thinking off through request body enable_thinking=false (chat template default is also off). --model gemma4-26b-a4b-capyctl-nvfp4-vllm --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 8 --server-slots-source capyctl-recipes@6ecf465 vllm/gemma-4-26b-a4b-capyctl-nvfp4-gb10/deployment.yaml:5-28 sets no max_num_seqs; vLLM 0.32.0 engine/arg_utils.py:2927-2931 default max_num_seqs 1024 for a 128 GB device (OPENAI_API_SERVER); CapyCTL router keeps at most 32 requests in flight (edurdias/capyctl issue 240); engine log: GPU KV cache 2,460,946 tokens, max concurrency 18.78x at 131,072 --server-ctx 131072 --server-ctx-source card GB10 box 2-gemma4-26b-a4b-capyctl-r3-nvfp4-vllm0320-mtp3-capyctl0112-1x-1911 field ctx
Quality v2

Quality

Not run on this setup: no Quality v2 test, so the page shows no quality number.

Credits

Also made this possible

  • Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistantGoogleHugging Face
  • Engine (vLLM)vLLM projectGitHub
Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages)
Model
capyctl/Gemma-4-26B-A4B-NVFP4 @241dbe07
Gemma 4 26B-A4B (base model) and the MTP drafter google/gemma-4-26B-A4B-it-assistant
huggingface.co/google/gemma-4-26B-A4B-it
Engine
edurdias' recipe as published (capyctl-recipes at commit 6ecf465); Speed v2 only — the program that loads the model and answers requests · github.com/vllm-project/vllm

Steps

  1. Get the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory.

    bash <start script of the run>
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
vLLM 0.32.0 (torch 2.13.0+cu130) through CapyCTL 0.1.12 standalone, Google MTP drafter depth 3 (google/gemma-4-26B-A4B-it-assistant @6e5aaaf4), lmcache off; bash <start script of the run> (starts CapyCTL 0.1.12 with the recipe vllm/gemma-4-26b-a4b-capyctl-nvfp4-gb10 of capyctl-recipes at commit 6ecf465: vLLM 0.32.0, checkpoint capyctl/Gemma-4-26B-A4B-NVFP4 r3, MTP drafter google/gemma-4-26B-A4B-it-assistant with 3 speculative tokens, FP8 KV cache, context 131072)

Context and sampling: 131,072 tokens (the recipe's context_length), FP8 (E4M3) KV cache, the recipe's parsers, sampling of the Gemma 4 card (temperature 1.0, top_p 0.95, top_k 64), seed 42, thinking off via the request body, 8 server slots (entered in our run card; the recipe sets no value for it).. Hardware: 1x NVIDIA DGX Spark (GB10, 128 GB unified memory). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages). All raw files of this setup: raw/v2/gemma4-26b-a4b-1spark-capyctl-r3-nvfp4-vllm-mtp3-capyctl-recipe-edurdias.