gnahzhang's LSQ3 recipe: DeepSeek-V4.1-Flash-1017

What it builds as an agent

Task: Voxel world (3D scene, one HTML file). One attempt, no help. Counted: the final file counts.

The browser check

• page loads without page error1 / 1
• window.ready === true within 30 s1 / 1
• no console errors (whole session)1 / 1
• frame is not blank1 / 1
• frame rate in software GL (rAF over 5 s)0 / 1
• world: scene size 0.5 (parts >= 80 or >= 20000 triangles; canvas fallback), visible text English if any 0.51 / 1
• automatic camera tour: the view changes on its own in two successive ~5 s windows (1 each)2 / 2
• animation in the world (nodes / instances / particles moving besides the camera)1 / 1
• OrbitControls: drag and wheel change the picture beyond the animation baseline0 / 1

Measured 2026-10-20. Raw files: run info · command · final choice · contract · sampling · check · screenshot · web.mp4 · web-poster.jpg.

Browser check 8.0 out of 10, measured on a second machine: the check that ran together with the agent run did not start, because the Docker VM on the Mac has 4 CPU cores and the check needs 16, so the run's own contract-show.json lists no score. I re-ran the same check on a Linux box with 16 or more cores from a copy of the run's final index.html (same sha256, checked by the tool) and took over only its score; its check.json is the evidence. The page loads, runs without console errors, is not blank, shows a world, flies an automatic camera tour and animates; the 2 lost points are the frame rate (4.77 frames per second in software rendering without a GPU, 20 are needed) and the orbit check (mouse drag and wheel changed the picture less than the check needs). The server's default of 16,384 tokens per answer was lifted to the card's 262,144 for every request. My test gate sent an empty keep-alive comment every 30 seconds so that the agent's connection was not cut after 300 seconds of silence; it changes nothing about model, prompt, sampling or output. The agent ended by itself after 65.7 minutes (89 requests, 168,228 tokens written); the “finish now” message (at 90 minutes) was not needed. One try, no help.

run_showcase.sh --harness dsh-021 --task VX --upstream http://127.0.0.1:64903/v1 --model deepseek-v41-lsq --sampling-file $RUN/sampling-show.json --out $RUN/raw/plan-2010-show-r2/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 1048576

How fast it runs

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median. Measured 2026-10-20. Raw files: speed result · speed contract · log · long answer.

Measured on profile 'benchmark' (1 slot, no prompt cache); decode with 2 and 4 requests at once does not apply (1 slot). The prompt for code ran at 67.7 tok/s, the prose prompt at 60.0 tok/s (medians of two runs each); the headline 63.83 tok/s is the mean of the two. The acceptance rate of the DSpark drafts is not available: oMLX reports no draft statistics.

python3 speed2.py --base-url http://127.0.0.1:60587/v1 --label run-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/plan-2010-speed-r0/work/speed2 --contract $RUN/raw/plan-2010-speed-r0/work/speed2/speed-contract.json --parts decode,prefill,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --tokenizer-json $HOME/models/ds41-1017-lsq3-g128-mlx/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 1.0, "top_p": 1.0} --sampling-source Same setting for thinking and non-thinking, no separate profile: deepseek-ai/DeepSeek-V4.1-Flash-1017 README.md:77-84 'Recommended sampling parameters' temperature 1.0, top_p 1.0 (copy not published; thinking mode only in the encoding, README.md:69); checkpoint has no generation_config.json; Baekpica README.md:147-148 ('DeepSeek recommends temperature 1.0 and top_p 1.0 for 1017'); server defaults T=1.0/top_p=1.0/min_p=0.05 at crates/ds4-server/src/parse.rs:18-20 --model deepseek-v41-lsq --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-20.

Note from the run: 137 of 400 MMLU-Pro answers and 149 of 198 GPQA Diamond answers were cut off at the 48-token limit (the model often starts with an explanation instead of the letter) and count as wrong. 1 of 40 LiveCodeBench answers was cut off at its limit of 32,768 tokens, 8 of 80 long-text answers at their limit, and 4 of 30 GPQA answers with thinking at the limit of 16,384 tokens; these count as wrong too. The author's model card says it has no quality measurement for the 1017 checkpoint, so these numbers are my own tests.

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts mmlupro --job plan-2010

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts gpqa --job plan-2010

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts ifeval --job plan-2010

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts lcb --job plan-2010

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts ruler --job plan-2010

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts ruler --job plan-2010

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts think --job plan-2010

How much context fits

Memory test. The recipe fixes the context (1,048,576 tokens), so the server is started once with the recipe's own command and reserves its memory up front. Then, for each size (4K, 8K, 16K ... up to the largest size tested), one prompt that fills about 95 % of that size is sent, and the model must answer. The largest size that answered counts; it is the highest size tested, not a limit of the setup. Decode speed is measured on every answer. Peak memory is the rise of used memory over the idle value before each server start (unified memory, so GPU and CPU share it). Measured 2026-10-20. Raw files: result of every step · log · contract.

Note from the run: fit memory per node, highest rise over idle: local 227.3 GiB The list of sizes ends at 128K (131,072 tokens), so this is the largest size tested, not the largest size the setup can load: the launcher's window is 1,048,576 tokens, and the author tested only 32,768 tokens of input. Decode at the six sizes ranges from 37.0 to 64.1 tok/s (45.1 tok/s at 128K); the answers are short (96 tokens at 4K, then up to a few hundred). The memory number is the rise over the idle value (wired + active + compressed, unified memory) at the largest step; the highest rise over all six steps was 227.3 GiB (at 16K).

bash tools/quality2/measure_all.sh tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json --parts fit --job plan-2010

Also made this possible

• ModelDeepSeekHugging Face @deepseek_ai GitHub Website
• Engine (oMLX)oMLX (jundot and contributors)@jundotkim GitHub Hugging Face Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

Mac Studio M5 Ultra 256 GB

System

macOS (the Mac above)

Model

DeepSeek-V4.1-Flash-1017 LSQ3-G128-MLX (gnahzhang)

Engine

oMLX on a Mac Studio, LSQ3-G128 quant (3-bit experts) with DSpark drafts — the program that loads the model and answers requests · github.com/jundot/omlx

Recipe and LSQ3 quant (LSQ3-G128, MLX)

huggingface.co/gnahzhang/DeepSeek-V4.1-Flash-1017-LSQ3-G128-MLX by gnahzhang

Steps

1. Get the recipe and the engine from the list above. The list has no direct download link for the model file itself. The base model is huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8011.

   Copy

   runtime.py serve \
     --work-dir ds41-lsq3-work-2010 \
     --model-dir MODELS_FOLDER \
     --profile cached \
     --port 8011

   The file names are the ones we used. Run the command in the folder that holds them, or put the full path in front of each name. Replace MODELS_FOLDER and BASE_FOLDER with your own folders.

   What the parts mean

   • --model-dir MODELS_FOLDERthe folder with the model files
   • --port 8011the network port the server listens on

oMLX d4298ad1 + omlx-flash-optimizations 29fe398 Patches 0001-0005 + Bundle-Patch 0006, MLX 59d600b5 (Quell-Build, Metal-JIT), DSpark 4 Draft-Tokens; runtime.py serve --work-dir $HOME/ds41-lsq3-work-2010 --model-dir $HOME/models/ds41-1017-lsq3-g128-mlx --profile cached --port 8011

Mia's own benchmark, run unchanged

This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section here and its own column on the class page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.

Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-20. Version 2.50.0, commit 21a3511.

QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.

Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.

Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.

Note from the run: sparkDash sends a request field called thinking_mode. This server's chat template accepts only chat and thinking, so a small relay on the Mac removed that one field from 454 of the 501 requests. Thinking stayed off through enable_thinking. Nothing else in the requests was changed.

python3 tools/quality2/sparkdash_client.py tools/quality2/cards/ds41-1017-lsq3-g128-omlx-cached.json <box> --client <client machine> --job plan-2010 --out $RUN

tool-eval-bench --base-url http://127.0.0.1:<port> --model deepseek-v41-lsq --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/work/tooleval/result.json --no-live