Ash's TensorFold recipe: NVIDIA Nemotron 3.5 Lightning 30B-A3B (MLX 4-bit + MTP drafts) on one DGX Spark

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-10. Raw files: speed result · speed contract · log.

Requests at once: the server has 8 slots (the engine's default, up to 8 streams; the recipe sets no value). The speed test ran with Ash Hart's recipe for this model (docs/recipes/nemotron-3.5.md, section Run, lines 16 and 17; release v1.2.5): tensorfold serve TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit --name bench. The weights are the TensorFold team's own MLX conversion of NVIDIA's BF16 checkpoint (affine 4 bits, group size 64: 4 shards and the MTP head file mtp-4bit.safetensors, about 17 GiB, revision d9d758fb), not a quant of mine. I started the engine from a folder with those files: on the first DGX Spark they were downloaded from Hugging Face at that revision, on the second DGX Spark they were copied from an archive copy of the same files; on both DGX Sparks they were checked against the Hugging Face sizes and sha256 hashes. The engine is the official release archive tensorfold-1.2.5-linux-aarch64.tar.gz (tag v1.2.5, sha256 1a0192b2; its binary tensorfold-native, sha256 4f7ed2ab), not built by me; it runs as a plain process, no container.

Which machine ran what: the quality tests and the memory test ran on the first DGX Spark. This speed test ran on a second DGX Spark, an identical machine (same GPU clock cap, same engine file, same weights). The speed part was run twice. The first run, on the first DGX Spark, is not used for any number on this page: its two runs of the code prompt decoded at 132.4 and 399.2 tok/s, a spread of 71.0 % against the 15 % limit of our tool, so no median could be trusted as a headline. Our analysis of that run's log found no cold start and no foreign load; one of the two code runs had copied a block of code out of its own prompt, which the engine's copy drafts match almost every time, so that run decoded at a far higher rate than the other (draft tokens accepted: 281 of 490, 57.3 %, in the first code run and 475 of 492, 96.5 %, in the second). The raw files of that first run are published too (speed result · speed contract · log); the numbers above are in them. I repeated the speed part on the second DGX Spark with the same engine release, the same weights and the same test settings, and that repeat is what this page shows: its spread is 2.3 % for the code prompt and 1.2 % for the prose prompt.

Differences from the author's recipe, all named: (1) A window of 262,144 tokens (--context 262144), the window of the checkpoint (its config.json), instead of the engine's default. Without --context the engine's own start line shows a window of 16,384 tokens (a start probe on the first DGX Spark; that log is not published), and the recipe's CUDA section names 16,384 tokens as the default requested context (line 82); the author's own measurements use --context 32768 (line 157). The context steps of 16K to 128K and the 32K prefill need a larger window: a prompt of 128K tokens plus its answer does not fit into 16,384 or 32,768 tokens. This is forced by my tests. (2) The streams: the recipe sets no --parallel, so the engine's default --parallel auto applies, which asks for up to 8 streams as memory and the engine allow (README, lines 104 and 343). The README tells you to read the accepted count at startup: the start line of this run reads '8 streams at once' (it is quoted in the engine_version field of the published speed2.json). So the page counts 8 slots. The label 'recipe' for the slot count in the raw files is the test tool's name for a value taken from the setup, not a setting of the recipe. (3) Changes of the command that do not change the numbers: a local folder instead of the Hugging Face name, --port 8080, --name nemotron-3.5-lightning instead of bench, and I start the release's binary tensorfold-native directly. (4) The recipe page measures with greedy decoding (temperature 0) and short replies of 96 to 256 tokens (lines 157 and 211), on Apple silicon; it gives no figure for a DGX Spark. I measure with the model card's sampling, 512 tokens and separate code and prose prompts, so the author's figures and these are not comparable, and the page makes no statement about them. Text only: no vision, no tool use and no agent task is measured.

The headline 141.9 tok/s is the mean of the code prompt (141.6) and the prose prompt (142.2), medians of two runs each. Thinking was off in every request, and the tool's cache check found no cached prompt tokens in any of the 60 requests it examined. The drafts were accepted at a rate of 60.8 % of the drafted tokens (1,182 of 1,943), 2.4 tokens per step on average. Requests at once: 1, 2, 4 and 8 requests ran together. The total rises from 132.6 tok/s with 1 request to 248.0 tok/s with 8; the speed of one request falls from 132.6 to 33.1 tok/s (the 1-request figure is measured over the whole request, so it is below the decode speed).

Load on the machine: the load guard (a sample every half second, 828 samples) discarded and repeated none of its 37 steps. The quiet check (one sample every 10 seconds, 42 samples) saw at most 1.8 % of one core, from the test runner's own script, and the quiet logs before and after the run list no foreign problems. Not published: the quiet logs and the sampling file of this run (the site's filter holds back files of that kind by name; the quiet logs carry GPU power readings), the load guard's sample file and error file, and the test tool's server logs. So the foreign-load figures above can be checked only in part: the speed contract, with the summaries of the load guard and of the quiet check, is published, while the guard's sample file, the quiet logs and the sampling file are not.

python3 speed2.py --base-url http://127.0.0.1:45541/v1 --label nea2-1110-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/nea2-1110-speed-r0/nea2-1110-speed-r0/speed2 --contract $RUN/nea2-1110-speed-r0/nea2-1110-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/tf-1.2.0-627/model/nemotron-lightning/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 1.0, "top_p": 0.95, "min_p": 0.0} --sampling-source nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 README 'Recommended Sampling: Temperature 1.0, Top_P 0.95' (README:59, :417) = generation_config.json of the MLX checkpoint (temperature 1.0, top_p 0.95). No top_k, no presence penalty. Thinking is a chat-template switch (enable_thinking, README:58). Seed 42 for speed/quality. --model nemotron-3.5-lightning --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 8 --server-slots-source server startup line of TensorFold 1.2.5 (start line read on the first DGX Spark on Nov 10, 2026); the recipe command sets no --parallel --server-ctx 262144 --server-ctx-source TensorFold 1.2.5 server default (start line, read on the first DGX Spark on Nov 10, 2026); recipe nemotron-3.5.md:15-18 sets no --context; raised with --context 262144

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-10.

Note from the run: MMLU-Pro (400 questions), GPQA Diamond (198), IFEval (200), LiveCodeBench (40 tasks) and RULER at 8K and 32K tokens (40 tasks each) ran with thinking off and the sampling of the model card (temperature 1.0, top-p 0.95; nothing else is named, so the engine's defaults apply), seed 42. Eight requests ran at once (the server's streams; each question is its own request with the same limits and seed). MMLU-Pro and GPQA Diamond ran with an answer limit of 1,024 tokens (the other pages use 48), letter only; IFEval with 4,096 tokens; LiveCodeBench with 32,768 tokens; RULER with 128 tokens. Without thinking the model answers the multiple-choice questions with a letter at once (median 2 tokens); 2 MMLU-Pro answers and 1 GPQA answer had no letter that could be read (they count as wrong). 1 IFEval answer and 7 LiveCodeBench answers were cut off at their limit and count as wrong; the other tests had none cut off. RULER by task at 8K and 32K: multi-key retrieval 10 of 10 and 10 of 10; common-word extraction 9 of 10 and 6 of 10; multi-hop question answering 7 of 10 and 4 of 10; variable tracking 2 of 10 and 2 of 10. The thinking test (GPQA 30) ran with thinking on and a limit of 16,384 tokens: 12 of the 30 answers were cut off at the limit and count as wrong (the model thinks long: median 6,873 tokens and 194.5 s per question, with 8 questions running at once). These quality tests have no foreign-load guard. Their sampling files (not published, see the speed notes) list the busiest other process every 10 seconds: during MMLU-Pro, GPQA, IFEval and RULER (66 samples) it was at most 0.8 % of one core, during the thinking test (105 samples) at most 2.6 %; during LiveCodeBench, 3 of 124 samples were above 25 % of one core: 1 of them a system information script of the operating system (100 %) and 2 the code sandbox of the LiveCodeBench test itself. GPQA Diamond: the original is gated and no Hugging Face token was available, so the public mirror fingertap/GPQA-Diamond (198 rows) was used. Not published, because the site's filter held them back: the IFEval answers (ifeval.jsonl; a model answer contains a word the filter reads as a machine name), the LiveCodeBench results file (lcb.jsonl; model code contains a number written like a power reading) and the contract file of the thinking test (it names the account of the machine in its sampling summary). The summaries and logs of all tests, and the LiveCodeBench generations (lcb.gen.jsonl), are published, and the numbers of the table were counted from the complete files. All quality tests ran on the first DGX Spark (see the speed notes). These are my own tests, and no other run of this setup is mixed into the numbers on this page.

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts mmlupro --job neb-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts gpqa --job neb-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts ifeval --job neb-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts lcb --job ned-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts ruler --job neb-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts ruler --job neb-1010

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts think --job nec-1010

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed at the largest size was not measured. Peak memory is the rise of used memory (MemTotal - MemAvailable) over the idle value before the step, on the one box. Unified memory, so GPU and CPU share it. Measured 2026-11-10. Raw files: result of every step · log · contract.

Note from the run: In this test the server was started again for each size, with its window set to that size (4,096 up to 131,072 tokens; the engine reserves its memory at the start). Then one prompt of about 95 % of the window (4K, 8K, 16K, 32K, 64K and 128K) was sent, and the model had to answer. All six sizes were answered (finish reason stop). The largest size tested is 131,072 tokens (a prompt of 124,518 tokens): it is the highest size I tested, not a limit of the setup. This test never used the 262,144-token window of the speed and quality runs, so there is no memory number for that window and no result above 128K. The memory number is the rise of the DGX Spark's used memory over its idle value before each server start (the GB10 has one pool of memory for CPU and GPU): the rise was 23.44 GiB with the 4K window and 31.66 GiB with the 128K window. Because it is the rise of the whole machine, other programs that used or released memory at the same time are inside the number. This engine reports no timings for this memory test, so the decode speed and the first-token columns are empty. The six server logs of this test (one per size) are not published, because the site's filter held them back (they contain German text from our start script); fit.json, the test's output and its contract are published.

bash tools/quality2/measure_all.sh tools/quality2/cards/nemotron-3.5-lightning-30b-a3b-mlx4bit-tf-1.2.5-official-1x-1010.json --parts fit --job nee-1010

How it plays the Gauntlet

One solo race, season seed 21, one attempt, no help.

Game rank as of Nov 10, 2026. The rank is the order of the Gauntlet finish times in this memory class. Race page and raw files.

The race ran the recipe named on its race page, with the sampling and answer limit of its lane config. The engine build of the race is not recorded in the race files.

Result (result.json) Race log (log.jsonl) Model answers (io.jsonl) System prompt Exact command Verify output Model config Race page

Also made this possible

• Engine (TensorFold 1.2.5, official release)Ash Hart@ashxhart GitHub Hugging Face Website
• Model (Nemotron 3.5 Lightning 30B-A3B)NVIDIAHugging Face GitHub Website
• MLX 4-bit weights (revision d9d758fb, with the MTP head)TTensorFold (Hugging Face organisation)Hugging Face

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1 NVIDIA DGX Spark (GB10, 128 GB unified memory), GPU clock capped at 2200 MHz; two identical DGX Sparks took part, see the speed notes

Model

TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit @d9d758fb (NVIDIA Nemotron 3.5 Lightning 30B-A3B, hybrid Mamba-2 + attention + MoE, 3B active; MLX affine 4-bit group 64 incl. MTP head)

Model (Nemotron 3.5 Lightning 30B-A3B)

huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

MLX 4-bit weights (revision d9d758fb, with the MTP head)

huggingface.co/TensorFold/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-MLX-4bit

Engine

TensorFold 1.2.5, the official Linux release (native CUDA engine, one GB10), MLX 4-bit weights with the MTP drafts of the checkpoint, 262,144 tokens of context (set by me, see the changes below), up to 8 streams (the engine's default) — the program that loads the model and answers requests · github.com/ashhart/TensorFold/releases/tag/v1.2.5

Recipe (docs/recipes/nemotron-3.5.md)

github.com/ashhart/TensorFold/blob/v1.2.5/docs/recipes/nemotron-3.5.md by Ash Hart

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8080.

   Copy

   tensorfold-native serve nemotron-lightning \
     --port 8080 \
     --name nemotron-3.5-lightning \
     --context 262144

   The file names are the ones we used. Run the command in the folder that holds them, or put the full path in front of each name.

   What the parts mean

   • --port 8080the network port the server listens on
   • --name nemotron-3.5-lightningthe model ID the server advertises to clients

TensorFold 1.2.5 official release archive tensorfold-1.2.5-linux-aarch64.tar.gz (sha256 1a0192b28d6ab6ec2749f3ff25515379dba4895171e781b30f810afc971ae4f8; tensorfold-native sha256 4f7ed2ab...), native CUDA family (nemotron), 1 GB10 (sm_121), MTP drafts, server default streams; tensorfold-native serve $HOME/tf-1.2.0-627/model/nemotron-lightning --port 8080 --name nemotron-3.5-lightning --context 262144 (Ash's recipe docs/recipes/nemotron-3.5.md, section Run, plus the changes named on this page; the folder holds the files of the Hugging Face repository at revision d9d758fb)