Ripple-65B Preview (BF16 with DFlash2 drafts) on two DGX Sparks, RicursiveAI's server script with SGLang 0.5.19

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-05. Raw files: speed result · speed contract · log · long answer.

Requests at once: the server has 16 slots (from the author's recipe). The speed test ran with the server script of RicursiveAI for the BF16 model (inference/server/serve.py at revision e27e1717 of the BF16 repository), imported unchanged: SGLang 0.5.19 with torch 2.13.0+cu130, transformers 5.12.1 and flashinfer-python 0.6.18 in a virtual environment on both boxes, the author's model code (RicursiveForCausalLM, BF16 only) and parsers, the BF16 weights (129.57 GB) and the DFlash2 draft model (8.08 GB). Both were checked file by file against the Hugging Face hashes on both boxes. The author's script is written for one machine with two B200 cards (the one speed figure on the BF16 model card is "reaching 250 tok/s on 2xB200 batch=8 prior to quantization or custom kernels"). The author has no recipe for two DGX Spark boxes and none for the BF16 model on a DGX Spark, so this setup is an adaptation. The differences from the script are: 2 boxes (--nnodes 2, tensor parallel 2 over the CX7 cable with NCCL) instead of one machine; the attention backend flashinfer, which is the backend of the author's DGX Spark recipe, instead of trtllm_mha with page size 64 (I did not compare the two); --mem-fraction-static 0.78 instead of 0.80; --disable-radix-cache; the loading arguments of the author's DGX Spark recipe (no memory mapping, drop the file cache after loading, no multi-thread loading); seed 42; a watchdog timeout of 1800 s and a distributed timeout of 3600 s; the autotune step for fp8_gemm skipped; and the two fixes for the GB10 chip from the author's DGX Spark code (spark.py lines 97 to 125), built in unchanged. The author's own values for the context (1,048,576 tokens), the slots (16) and the drafts (16 tokens, draft window 16,384) are not changed. This server takes 16 requests at a time and drafts of 16 tokens; the page for one DGX Spark uses 1 request at a time and drafts of 32 tokens (the values of the author's DGX Spark recipe). Weights, number of boxes, slots and drafts all differ, so the two pages do not compare one to one. The context window is the default of the author's current server script (since its commit e27e1717 of 4 Nov 2026). At its start SGLang warns that 1,048,576 tokens is longer than the 131,072 tokens it derives from the model files and that this may lead to incorrect outputs; I kept the author's value. All four long-prompt steps fit; the 128K step is a prompt of 131,072 tokens. The author names no sampling anywhere; the author's own examples use temperature 0, and so does this test (seed 42). Thinking cannot be switched off: the chat template has no switch (the author's notebook says so), and the request flag for no thinking that the speed test sends is ignored. In the 51 chat requests of the warm-up, the four decode runs, the requests-at-once test and the long-prompt test (each asked for 256 or 512 tokens) the whole reply was thinking text, with no visible answer character; the long answer had both (92,782 characters of thinking text and 9,317 visible characters). The 9 prefill probes are plain 8-token completions of the filler log without the chat template. So the decode cells are the speed of the first 512 tokens of thinking text with drafts. Words per second and characters per second of the visible answer are 0 in the raw files, so the page does not show them and gives no time for 500 words. How to read the decode figure: it measures the start of the model's thinking text, not a finished answer, and how many of the draft model's guesses the main model accepts depends on the text. A separate diagnosis on one DGX Spark with the NVFP4 recipe found that short code answers decode faster than these cells; it is not part of the v2 rows, and none of its numbers is shown here. The headline decode of 10.11 tok/s is the mean of the code prompt (9.19) and the prose prompt (11.04), medians of two runs each (spread 0.8 % and 3.7 %). The speed varied between runs: the one-request rounds of the requests-at-once test ran at 9.9 and 10.6 tok/s in total (the table shows their median, 10.3), and the long answer averaged 14.9 tok/s over its decode window. The prose cell at 65,536 tokens (13.4 tok/s) is the median of two runs that differ a lot (10.3 and 16.5 tok/s, spread 32.7 %): the speed contract flags it as noisy ("re-measure"), and I did not run extra rounds for it. The speed test could not read draft statistics from the server (the author's script has no metrics flag), so the page shows no acceptance rate. The server has 16 slots, as in the author's script, so the requests-at-once test (1, 2, 4 and 8) can run in parallel: the total rises to 18.2, 31.4 and 53.0 tok/s for 2, 4 and 8 requests (one request: 10.3), and each request gets 10.3, 9.2, 8.2 and 6.9 tok/s for 1, 2, 4 and 8 requests. The long answer (the house prompt, up to 32,768 tokens) ran into the token limit after 2,203.3 s and counts as wrong. The server does not report cached prompt tokens, so the check for them could not confirm anything for the 61 requests; the recipe switches the prefix cache off. A first full run on 5 Nov 2026 (10:42 to 13:11) was discarded: the guard threw away three tries of the long answer because of foreign load (the Docker daemon, then twice the package manager's hourly check), and I ended one already disturbed try myself with the server's abort call. Nothing from that run is on this page. This page shows the second run (5 Nov 2026, 13:40 to 15:31). The load guard (a sample every half second, on the box that runs the test tool) saw other processes together above 25 % CPU in 150 of 11,065 samples, up to 108 %: mostly a firmware query tool (61 samples), package-manager checks (46), nvidia-smi calls (18), the test tool's own tokenizer helper (12), the box's update-check scripts (10) and container and firmware services (3). It threw away two tries and repeated them: the second round of the two-requests-at-once step (a package-manager check, up to 102 %) and the second code run at 65,536 tokens (a package-manager command, up to 102 %), both in the package manager's hourly window; the repeats are the ones that count, and no other step had to be repeated. The second box has no guard; its load log (a sample every ten seconds, not among the raw files) shows no foreign process above 12 %. The guard's sample file and the quiet-check file are not among the raw files here; the speed contract lists the guard's steps and the tries it threw away. The ten-second quiet-check file lists the SGLang scheduler itself as the busiest process in 549 of its 550 samples, so it hides most foreign load and the guard above is the record of it.

python3 speed2.py --base-url http://127.0.0.1:35973/v1 --label ripple65b-bf16-box-3-4-0511b-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/ripple65b-bf16-box-3-4-0511b-speed-r0/ripple65b-bf16-box-3-4-0511b-speed-r0/speed2 --contract $RUN/raw/ripple65b-bf16-box-3-4-0511b-speed-r0/ripple65b-bf16-box-3-4-0511b-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/ripple-65b-bf16-e27e1717/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.0} --sampling-source The author names NO recommended sampling anywhere (model cards BF16/NVFP4/DFlash2, launch blog, generation_config.json without temperature/top_p). His own examples run with temperature 0: inference/local/spark.py:175 (complete()), :187 (chat()), ripple.ipynb cell 3 (sampling_params temperature 0), inference/server/README.md tool example ('temperature': 0). top_p/top_k/penalties are not sent. The model always reasons: the chat template has no thinking switch (ripple.ipynb markdown 3; server/README.md:67), so 'instruct' and 'thinking' are the same sampling and enable_thinking has no effect. --model ripple --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 16 --server-slots-source ricursiveAI/Ripple-65B-Preview @e27e1717 inference/server/serve.py @e27e1717:43 (--max-running-requests 16); inference/server/README.md (send eight concurrent requests to each port for sixteen total) --server-ctx 1048576 --server-ctx-source inference/server/serve.py @e27e1717:43 (--context-length 1048576, the author's current BF16 server default since commit e27e1717 of Nov 4 23:02 UTC 'Set server context to 1Mi with a 16Ki draft window'); the server's own value is read at start (GET /v1/models max_model_len)

Quality

No quality test was run on this setup, so the page shows no value for it: no MMLU-Pro, GPQA Diamond, IFEval, long-text, LiveCodeBench or thinking value, no memory test and no sparkDash. Thinking cannot be switched off and the full set would take about 40 hours.

Also made this possible

• Draft model (Ripple-65B Preview DFlash2, revision 35609b3b)RricursiveAI/Ripple-65B-Preview-DFlash2Hugging Face
• Engine (SGLang 0.5.19)SSGLangGitHub

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

2x NVIDIA DGX Spark class boxes linked by CX7 RDMA (109.2 and 109.3 Gb/s on the two links, measured before the run): 1x Gigabyte AI TOP ATOM and 1x Lenovo ThinkStation PGX (GB10 chips, 128 GB unified memory each). Both run driver 580.178.04 and kernel 7.0.0-1019-nvidia, GPU clock capped at 2200 MHz (torch 2.13.0+cu130).

Model

ricursiveAI/Ripple-65B-Preview @e27e1717 (65B dense, 80 layers, SWA + global NoPE; BF16) + ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b (BF16 drafter, 4.04B)

Model, BF16 weights, DFlash2 draft model and the server script (inference/server/serve.py)

huggingface.co/ricursiveAI/Ripple-65B-Preview/tree/e27e171792f303d74c048184e8259ba663cfa0d5

Draft model (Ripple-65B Preview DFlash2, revision 35609b3b)

huggingface.co/ricursiveAI/Ripple-65B-Preview-DFlash2/tree/35609b3be3dd08a822b3ba1257c1ea5dd6f87632

Engine

SGLang 0.5.19 with the author's server script and model code, RicursiveAI's BF16 weights (revision e27e1717) and DFlash2 draft model (revision 35609b3b), tensor parallel over 2 boxes, 1,048,576 tokens of context, 16 requests at a time — the program that loads the model and answers requests · github.com/sgl-project/sglang

Steps

1. Get the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory.

   Copy

   python serve_ripple_2sparks.py

SGLang 0.5.19 (torch 2.13.0+cu130, transformers 5.12.1, flashinfer-python 0.6.18, venv), model code = author inference/server/model.py (RicursiveForCausalLM, BF16 only) + ripple_format.py, DFLASH speculative (16 draft tokens), flashinfer attention, TP=2 over 2 GB10 (sm_121, box-3 + box-4, CX7 direct cable, NCCL), context 1048576, 16 slots; python serve_ripple_2sparks.py (imports the author's inference/server/serve.py unchanged: DEFAULT_ARGS inference/server/serve.py @e27e1717:36-46; plus --nnodes 2 --node-rank 0|1 --dist-init-addr [address of the head box]:5757 --attention-backend flashinfer (page size default) --mem-fraction-static 0.78 --disable-radix-cache --weight-loader-disable-mmap --weight-loader-drop-cache-after-load --random-seed 42 --watchdog-timeout 1800 --dist-timeout 3600; run_scheduler_process_func = the author's DGX Spark fixes spark.py:97-125; wrapper $HOME/ripple65b-bf16-0511/ripple2-ctl.sh)