Home / 256 GB

256 GB · 2× DGX Spark (GB10)

Hy4-Flash 220B (FP8) on two DGX Sparks: ZackSample's community recipe on vLLM (thinking on)

Weights: FP8 (Tencent's own FP8 release tencent/Hy4-flash-FP8, revision 003107fa, unchanged), DFly2 drafter from the same release

2x NVIDIA DGX Spark (GB10, 128 GB unified memory each), tensor parallel 2 over two CX7 cable paths (RDMA check before the run: 109.32 and 109.35 Gb/s). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages) on both boxes

Community recipe (start script, four patch files and benchmark: ZackSample/hy4-flash-dgx-sparks at commit a0e88d7)ZackSample
vLLM 0.1.dev25463+g77766db1f (Tencent-Image vllm/vllm-openai:hy4-flash) + ZackSample-Patches (attention.py, sm120.py, hy4_attention.py, engram.py), TP=2 across 2 Knoten (nnodes 2, without Ray), DFly2 5 Draft-Tokenscommit ZackSample/hy4-flash-dgx-sparks a0e88d769cdb1f9f4bbfdc06e1a621a6a1ffe169; Image sha256:6b645bfbf4b89602d38f6e93c1c418516ec0bfbf54b64cd661b8075bfec01da12 GB10 boxes (CX7, TP2)measured 2026-11-19

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed v2

How fast it runs

Decode
15.9tok/s
Decode, code
14.3tok/s
Decode, prose
17.5tok/s
Prefill at 8K
1177.2tok/s
First token at 2K
1.8s
MeasurementResult
Decode15.9 tok/s
Decode, code prompt14.3 tok/s
Decode, prose prompt17.5 tok/s
Spread of the decode runs (CV)4.1 %
Spread of the code runs (CV)0.8 %
Spread of the prose runs (CV)4.1 %
Decode, window 119.2 tok/s
Prefill at 2K1129.8 tok/s
Prefill at 8K1177.2 tok/s
Prefill at 32K1172.8 tok/s
Time to first token at 2K1.8 s
Time to first token at 8K7.0 s
Time to first token at 32K27.9 s
Draft acceptance rate22.6 %
Mean accepted length2.1 tokens
1 request at once (total)17.8 tok/s
2 at once (total)23.9 tok/s
4 at once (total)22.9 tok/s
8 at once (total)23.4 tok/s
1 request at once (per request)17.8 tok/s
2 at once (per request)12.9 tok/s
4 at once (per request)8.1 tok/s
8 at once (per request)5.0 tok/s
Server slots (2 as set by the recipe)2 slots
Long answer, wall time692.4 s

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens1168.3 tok/s14.0 s13.8 tok/s17.8 tok/s
32K tokens1158.0 tok/s28.3 s14.5 tok/s19.6 tok/s
64K tokens1148.4 tok/s57.1 s14.5 tok/s18.6 tok/s
128K tokens1109.4 tok/s117.2 s16.5 tok/s16.5 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-19. Raw files: speed result · speed contract · log · long answer.

What ran. ZackSample's community recipe for Hy4-Flash on two DGX Sparks (ZackSample/hy4-flash-dgx-sparks at commit a0e88d769cdb1f9f4bbfdc06e1a621a6a1ffe169; author and repository owner ZackSample): Tencent's image vllm/vllm-openai:hy4-flash (vLLM 0.1.dev25463) with the four patch files of the recipe, Tencent's FP8 weights (tencent/Hy4-flash-FP8, unchanged) with the DFly2 drafter (5 drafted tokens), tensor parallel 2 over two nodes, FP8 KV cache (fp8_ds_mla), window 131,072 tokens, 2 slots, GPU memory utilisation 0.80, all as in the recipe's start script (start.sh lines 24 to 27). Deviations: the weights are read from a network share instead of a local disk, and the n-gram table (Engram) is a local file of equal bytes on each box (checked with cmp). The model loaded in 1,511 s; each rank holds 77.81 GiB of weights and the KV cache holds 436,140 tokens.

Thinking is on. Hy4 takes its thinking switch as reasoning_effort (high by default, or no_think), not as the flag enable_thinking that the standard test sends; so every decode, requests-at-once and context answer came as reasoning (512 of 512 tokens were reasoning, no visible answer text). That is why there is no words-per-second value and no 'about X seconds for 500 words' on this page. The decode numbers are the speed of reasoning tokens; they are not comparable with the Hy4 pages of the Maya setups, which run with thinking off.

The headline 15.9 tok/s is the mean of the code prompt (14.3) and the prose prompt (17.5), medians of two runs each. The code prompt has 1,214 and 1,216 tokens and the prose prompt 1,300 and 1,304 tokens; every reply is 512 tokens and ends at the limit. Accepted drafts: 22.6 % (2.13 mean accepted length).

Long prompts. Prefill at 2K, 8K and 32K tokens: 1129.8, 1177.2 and 1172.8 tok/s. Context steps (decode on the code prompt): 16K 13.8 tok/s (prefill 1168.3), 32K 14.5 (prefill 1158.0), 64K 14.5 (prefill 1148.4) and the 128K step 16.5 tok/s (prefill 1109.4 tok/s, first token after 117.2 s); the 128K step is measured at 130,048 tokens (the server window of 131,072 minus 1,024).

Requests at once. The recipe sets 2 slots, so 1 and 2 requests run in parallel and the levels 4 and 8 queue behind the 2 slots. The total is 17.8, 23.9, 22.9 and 23.4 tok/s with 1, 2, 4 and 8 requests and the speed of each request 17.8, 12.9, 8.1 and 5.0 tok/s.

Sampling. The speed test sent the sampling of the model card (temperature 0.9, top_p 1.0) with the flag enable_thinking false (which has no effect on Hy4, see above), seed 42 and cache_prompt false; two runs per prompt of 512 tokens, medians. Cache check: the check of the test tool is marked not ok: 0 of the 61 requests report cached tokens (the details are in the raw speed result).

Foreign load. The load guard repeated 5 of 38 steps (7 repeats); details in the guard section of the speed result.

The author's own numbers: ZackSample's README lists, measured on 18 November 2026 with his own benchmark (bench.py), 15 tok/s for prose, 30 for code and 23 for thinking, and a prefill of 810 to 1,050 tok/s. His benchmark is not this page's standard test; the numbers are listed as his, not measured by us here.

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run hy4-flash-fp8-2sparks-zs-1911-speed-r0)
python3 speed2.py --base-url http://127.0.0.1:42449/v1 --label hy4-flash-fp8-2sparks-zs-1911-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/hy4-flash-fp8-2sparks-zs-1911-speed-r0/hy4-flash-fp8-2sparks-zs-1911-speed-r0/speed2 --contract $RUN/raw/hy4-flash-fp8-2sparks-zs-1911-speed-r0/hy4-flash-fp8-2sparks-zs-1911-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 0.9, "top_p": 1.0} --sampling-source Tencent HF-card tencent/Hy4-flash: 'Recommended sampling parameters: temperature=0.9, top_p=1.0' and generation_config.json (do_sample true, temperature 0.9, top_k -1, top_p 1) --model hy4-flash --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 2 --server-slots-source ZackSample/hy4-flash-dgx-sparks@a0e88d7 start.sh:25 (PARALLEL=2) and :59 (--max-num-seqs); C4/C8 queue behind the 2 slots, stated openly --server-ctx 131072 --server-ctx-source card box-1-hy4-flash-fp8-vllm-zacksample-tp2-2x-a0e88d7-1911 field ctx
Quality v2

Quality

Not run on this setup: no Quality v2 test, so the page shows no quality number.

Credits

Also made this possible

  • Model, FP8 weights (tencent/Hy4-flash-FP8) and the DFly2 drafter that ships with themTencentHugging Face
  • Engine (vLLM, Tencent's image vllm/vllm-openai:hy4-flash)vLLM projectGitHub
Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
2x NVIDIA DGX Spark (GB10, 128 GB unified memory each), tensor parallel 2 over two CX7 cable paths (RDMA check before the run: 109.32 and 109.35 Gb/s). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages) on both boxes
Model
Tencent Hy4 Flash (220B, about 12B active) FP8 (tencent/Hy4-flash-FP8 @003107fa, unchanged) + DFly2 drafter (in the checkpoint)
Model, FP8 weights (tencent/Hy4-flash-FP8) and the DFly2 drafter that ships with them
huggingface.co/tencent/Hy4-flash-FP8
Engine
Tencent's FP8 weights and DFly2 drafter, tensor parallel 2; the decode answers are all reasoning; Speed v2 only — the program that loads the model and answers requests · github.com/vllm-project/vllm

Steps

  1. Get the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory.

    bash <start script of the run>
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
vLLM 0.1.dev25463+g77766db1f (Tencent-Image vllm/vllm-openai:hy4-flash) + ZackSample-Patches (attention.py, sm120.py, hy4_attention.py, engram.py), TP=2 across 2 Knoten (nnodes 2, without Ray), DFly2 5 Draft-Tokens; bash <start script of the run> (two ranks: docker run <image vllm/vllm-openai:hy4-flash with the four patch files of the recipe> vllm serve <Hy4-flash-FP8 weights> --served-model-name hy4-flash --tensor-parallel-size 2 --nnodes 2 --kv-cache-dtype fp8_ds_mla --max-model-len 131072 --max-num-seqs 2 --gpu-memory-utilization 0.80; the recipe's start script sets the DFly2 drafter with 5 tokens)

Context and sampling: 131,072 tokens per request, 2 parallel requests (the recipe's start script), GPU memory utilisation 0.80, DFly2 drafter with 5 tokens, KV cache of 436,140 tokens, FP8 KV cache (fp8_ds_mla). Thinking is on (the standard test's flag enable_thinking false has no effect on Hy4); sampling of the model card (temperature 0.9, top_p 1.0), seed 42.. Hardware: 2x NVIDIA DGX Spark (GB10, 128 GB unified memory each), tensor parallel 2 over two CX7 cable paths (RDMA check before the run: 109.32 and 109.35 Gb/s). Driver 580.178.04, kernel 7.0.0-1019-nvidia (4 KiB pages) on both boxes. All raw files of this setup: raw/v2/hy4-flash-fp8-2sparks-vllm-zacksample-tp2-a0e88d7.