Mia's Nemotron 3.5 Bolt recipe on one DGX Spark: NVIDIA Nemotron 3.5 Bolt 74B-A9B

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-02. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 16 slots (from the author's recipe). The speed test ran with Mia's recipe unchanged (MiaAI-Lab/Nemotron-3.5-Bolt-TensorFold-DGX-Spark, commit 6b064a31 of 1 Nov 2026, the head of main on 2 Nov 2026): TensorFold v1.1.6 (commit eea9dc9c) with her patch 0002 and libbolt, built from source by the recipe's own prepare.sh, her defaults for context (262,144 tokens), 16 streams, an 8 GiB prompt cache and vision on. Only names differ from her start command and change no number: port 8897, the server listens on localhost only, and the build script's cleanup step was replaced by 'do nothing' (my rule: no deletion). Sampling is the model card's: temperature 1.0, top_p 0.95; seed 42; the request flag for no thinking; no cached tokens. The headline 67.92 tok/s is the mean of the code prompt (63.38) and the prose prompt (72.45), medians of two runs each (spread 3.6 % and 1.2 %). Drafts were accepted at a rate of 35.5 %. Mia's README lists, for sampled decoding (temperature 1.0, top_p 0.95) with one client, 71.8 tok/s for prose and 136.1 tok/s for code (TensorFold's own bench tool, 256 tokens, median of 3). My prose prompt gives 72.45 tok/s, 0.9 % above her prose number. My code prompt is another prompt than hers (my result 63.38 tok/s against her 136.1), so the two code numbers are not comparable. The 128K prose decode (66.3 tok/s) is the median of two runs that differ widely (57.7 and 74.9 tok/s); the raw file flags it as noisy. Open issue on this engine and box type: TensorFold #1654 / Mia #4 (HTTP 500 from the Bolt forward pass under sustained load with 2 or more parallel clients on GB10); it did not occur in this speed run. The quality tests for this setup have not been run yet. Other processes used up to 150 % CPU during the speed test (the speed contract names the busiest one); the load guard threw away and repeated 2 steps, and according to the guard every value kept is from a clean try.

python3 speed2.py --base-url http://127.0.0.1:37897/v1 --label bolt-box-1-mia-0211-r1-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/bolt-box-1-mia-0211-r1-speed-r0/bolt-box-1-mia-0211-r1-speed-r0/speed2 --contract $RUN/bolt-box-1-mia-0211-r1-speed-r0/bolt-box-1-mia-0211-r1-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/.cache/huggingface/hub/models--nvidia--NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB/snapshots/a326822af111290f2af5ac59ef4b5040347dabc4/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 1.0, "top_p": 0.95} --sampling-source https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB README.md line 66 ('Text: temperature 1.0, top_p 0.95') @a326822a; local copy source/hf-bolt-24gb-README.md; Mia scripts/config.sh:90-93 --model Nemotron-3.5-Bolt --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 16 --server-slots-source Mia scripts/config.sh:59-61 (PARALLEL 1 to 16, default 16) = the recipe default, unchanged --server-ctx 262144 --server-ctx-source card spark-bolt74b-iq2-lilicorr-tensorfold-mia-6b064a31-1spark field ctx

Quality

Quality tests have not been run on this setup yet.

Also made this possible

• Model (24 GB pack, revision a326822a)NVIDIAHugging Face GitHub Website
• Draft model (LiLiCorr, revision 4c6a5613)NVIDIAHugging Face GitHub Website
• Engine (TensorFold v1.1.6, commit eea9dc9c)Ash Hart@ashxhart GitHub Hugging Face Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA DGX Spark (GB10, 128 GB unified memory), one box. Driver 580.178.04, kernel 7.0.0-1019-nvidia.

Model

nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB @a326822a + nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr @4c6a5613

Model (24 GB pack, revision a326822a)

huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB

Draft model (LiLiCorr, revision 4c6a5613)

huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr

Engine

TensorFold 1.1.6 as the recipe builds it (sm_121), 24 GB pack with IQ2_XXS experts and NVFP4 Mamba layers, LiLiCorr drafts, 262,144 tokens of context, 16 streams, vision on — the program that loads the model and answers requests · github.com/ashhart/TensorFold

Recipe (TensorFold on a DGX Spark), commit 6b064a31

github.com/MiaAI-Lab/Nemotron-3.5-Bolt-TensorFold-DGX-Spark by Mia's AI Lab

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8897.

   Copy

   PORT=8897 HOST=127.0.0.1 ./start.sh

   What the parts mean

   • PORT=8897the network port the server listens on

TensorFold v1.1.6 tensorfold-native (Zig, CUDA, sm_121a) + Mia Patch 0002 + libbolt (sm_121a), LiLiCorr drafts, built from source in Docker by the recipe's prepare.sh; PORT=8897 HOST=127.0.0.1 ./start.sh