niklaslenzProject Maya: DeepSeek-V4.1-Flash-1017 (Maya-S)
1x NVIDIA GeForce RTX 5090 32 GB, 96 GB RAM, 4 TB NVMe SSD (expert tier: most of the 340 GB model is offloaded to RAM and NVMe)
Overall score as of 23 Oct 2026. Scores are relative to the best run in the class today; a better model later moves older runs down.
How the overall score is calculated
| Part | This run | Best in class | Relative | Weight | Points |
|---|---|---|---|---|---|
| Speed, one request | decode 20.9 tok/s | best 224.9 tok/s | 9.3 % | 17.5 % | 1.6 |
| Speed, 8 at once | total 19.7 tok/s | best 198.5 tok/s | 9.9 % | 17.5 % | 1.7 |
| Quality | mean of 7 tests: 68.0 % | best mean 68.0 % | 100.0 % | 25 % | 25.0 |
| Browser check | 9 of 10 | 90.0 % | 5 % | 4.5 | |
| My rating | 6 of 10 | 60.0 % | 35 % | 21.0 | |
| Overall | 100 % | 53.9 |
The seven quality tests: MMLU-Pro 70.8 · GPQA 52.5 · IFEval 88.0 · LiveCodeBench 60.0 · long text 8K 72.5 · long text 32K 52.5 · GPQA 30 with thinking 80.0.
The speed part is half the decode speed with one request and half the total speed with 8 requests at once, each relative to the best run of this class; a run without a total for 8 requests at once gets the better of its decode speed alone and the mix with its highest measured total. Quality is relative to the best run of this class, the browser check and my rating are out of 10. The parts are rounded one by one. sparkDash, the memory test and speculative-decoding numbers are not part of it. How the score is made, and the assumption behind it.
What it builds as an agent
Task: Airship dock (3D scene, one HTML file). One attempt, no help. Counted: the final file counts.
The exact promptcopy it and try it yourself↓
This is the task prompt word for word, as we handed it to the agent harness. The harness adds its own system prompt and tools, and some harnesses add short context notes of their own. A run that goes past 90 minutes also gets a short “finish now” message, as the prompt says. To try it yourself, give it to your own coding agent: it needs a shell and a working folder.
Design and build the most impressive, creative, elaborate and detailed steampunk airship dock voxel world you are capable of. Theme: a steampunk airship dock. Everything else (style, story, extra details) is your choice.
How you work:
- You work alone in the folder /work. Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes. Spend the time where it shows in the result.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh index.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- You may create your own assets locally with these tools (textures, sky, sounds, music) and embed them in the file as data URIs. Do not download anything.
- Keep a working /work/index.html at all times: write a first working version early, then improve it step by step. If you are stopped, the last version that loaded without errors counts.
- Check your work with `shot.sh` as often as you like, fix every error, and keep improving until you are satisfied. Then stop.
Deliverable: ONE self-contained file /work/index.html that opens and runs in Chrome.
- JavaScript libraries: only these, through an import map with these exact URLs (they work offline in the test browser). Use whatever helps, including all Three.js addons (post-processing, sky, water, noise and so on):
- "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js"
- "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/"
- "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js"
- "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js"
- "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js"
- No other external resources. Build every object from cubes (voxels); InstancedMesh is allowed.
- The page fills the whole window and resizes with it. All visible text is in English.
- After the scene is built, set `window.scene` to the Three.js scene and `window.ready = true`.
- An automatic slow cinematic camera tour starts on load. Mouse input switches to OrbitControls (drag to rotate, scroll to zoom).
- Sound, if any, starts only after the first click (browsers block autoplay).
- A small label in the top-left corner: "Steampunk Airship Dock".
Minimum content (go far beyond it):
1. A tall iron docking tower with platforms, stairs or ladders, and a mooring arm at the top.
2. At least two airships, each with a balloon envelope, a gondola and propellers.
3. Big brass gears on the tower or the buildings.
4. Pipes and at least three chimneys that emit rising steam particles.
5. At least one crane and a stack of cargo crates.
6. A ground area with a workshop building and a small rail track.
7. Propellers and gears rotate, steam rises and fades, and one airship slowly approaches the tower, docks, waits and leaves again, in a loop.
8. Warm sunset lighting with shadows.
Play it yourself
Open full screenAI-generated The page this setup built, running live in your browser: click into it and use the controls it shows itself (usually drag to look around, scroll to zoom). It needs WebGL and can be slow on a weak graphics card.
The browser check
- page loads without page error1 / 1
- window.ready === true within 30 s1 / 1
- no console errors (whole session)1 / 1
- frame is not blank1 / 1
- frame rate in software GL (rAF over 5 s)0 / 1
- required objects (voxel mass, >=3 steam emitters, >=2 airships, label)2 / 2
- motion: pixel diff 0.5, >=3 rotating nodes 0.5, large group translating 12 / 2
- OrbitControls: drag and wheel change the picture beyond the animation baseline1 / 1
Measured 2026-10-22. Raw files: check · run info · command · final choice · contract · sampling · gate stats · screenshot · clip.mp4.
Other processes used up to 1433 % CPU during the run. Browser check 9.0 out of 10: the page loads, runs without console errors, is not blank, shows the required objects (about 26,500 voxel-equivalents, steam sources, at least two airships and the label “Steampunk Airship Dock” at the top left), moves on its own and answers to mouse drag and wheel. The lost point is the frame rate: 2.17 frames per second in software rendering without a GPU (20 are needed). The agent (DeepSeek Harness 0.2.1-rc.2, the harness DeepSeek ships) worked 74.6 minutes, one try, no help. It did not finish by itself: the prompt of the last request had 125,831 of the 131,072 tokens of context; my test gate estimated it at 126,060 tokens and subtracted a safety margin of 10,341 tokens, so it cut the answer limit to the minimum of 16 tokens (the real room was 5,241 tokens). That answer was cut off, and the harness ended with an error (exit code 1). The last saved version of the page counts; it loads without errors (10 of 11 versions seen loaded cleanly). The “finish now” message (from 90 minutes) was not needed. 32 requests, no error, 98,217 answer tokens. The limit per answer was 262,144 tokens, the value on the DeepSeek model card; because the context is 131,072 tokens, my test gate lowered the limit to the room left in the context (minus a safety margin) in 24 of the 32 requests. Thinking was on, sampling temperature 1.0 and top_p 1.0, reasoning effort high (the author's default). About the sentence on other processes above: it comes from one sample of the load monitor (1,433 % for a process named “python3 -”, which I did not identify). In 452 of the 464 samples the busiest other process was the kernel's memory manager (kswapd0, 400 samples) or the test browser (52 samples). Maya runs this 340 GB model as an expert tier over the 5090's VRAM, 96 GB of RAM and an NVMe SSD, so most of the model is offloaded: the engine log of the setup start counts about 1,970 of 15,360 experts in VRAM, and about 13,390 run on the CPU lane and are read in place from the model's files. Mia's Qwen3.8-27B in the same 32 GB class runs fully in VRAM, so the two sets of numbers are not built the same way.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
run_showcase.sh --harness dsh-021 --task K1 --upstream http://127.0.0.1:36349/v1 --model deepseek-v4.1-flash --sampling-file $RUN/sampling-show.json --out $RUN/raw/quality-v2-221022-maya-v1122-show-r2/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 131072How fast it runs
| Measurement | Result |
|---|---|
| Decode | 20.9 tok/s |
| Words per second | 14.2 words/s |
| Characters per second | 93.2 chars/s |
| Decode, code prompt | 22.9 tok/s |
| Decode, prose prompt | 18.9 tok/s |
| Spread of the decode runs (CV) | 13.0 % |
| Spread of the code runs (CV) | 10.9 % |
| Spread of the prose runs (CV) | 13.0 % |
| Decode, window 1 | 18.5 tok/s |
| Prefill at 2K | 246.7 tok/s |
| Prefill at 8K | 828.3 tok/s |
| Prefill at 32K | 1847.5 tok/s |
| Time to first token at 2K | 8.3 s |
| Time to first token at 8K | 9.9 s |
| Time to first token at 32K | 17.7 s |
| 1 request at once (total) | 14.3 tok/s |
| 2 at once (total) | 18.9 tok/s |
| 4 at once (total) | 19.5 tok/s |
| 8 at once (total) | 19.7 tok/s |
| 1 request at once (per request) | 14.3 tok/s |
| 2 at once (per request) | 14.1 tok/s |
| 4 at once (per request) | 8.1 tok/s |
| 8 at once (per request) | 4.4 tok/s |
| Server slots (from the recipe) | 1 slots |
| Long answer, wall time | 363.9 s |
Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds; requests at once (later supplementary run of 2026-10-22, same setup): C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-10-22. Raw files: speed result · speed contract · log · long answer · speed result · speed contract · log.
Requests at once: the server has 1 slot (from the author's recipe). C2, C4, C8 exceed the slots: the extra requests wait in the queue, and they were measured anyway. Requests at once: the server has 1 slot (from the author's recipe). C2, C4, C8 exceed the slots: the extra requests wait in the queue, and they were measured anyway. The speed run itself measured 1 request at once (total) at 12.08 tok/s; the supplementary run, which measured all levels in one session, measured 14.34 tok/s, and that value is shown. The speed run itself measured 2 at once (total) at 19.03 tok/s; the supplementary run, which measured all levels in one session, measured 18.91 tok/s, and that value is shown. The speed run itself measured 4 at once (total) at 19.96 tok/s; the supplementary run, which measured all levels in one session, measured 19.47 tok/s, and that value is shown. The speed run itself measured 8 at once (total) at 19.8 tok/s; the supplementary run, which measured all levels in one session, measured 19.69 tok/s, and that value is shown. The speed run itself measured 1 request at once (per request) at 12.08 tok/s; the supplementary run, which measured all levels in one session, measured 14.34 tok/s, and that value is shown. The speed run itself measured 2 at once (per request) at 14.28 tok/s; the supplementary run, which measured all levels in one session, measured 14.09 tok/s, and that value is shown. The speed run itself measured 4 at once (per request) at 8.2 tok/s; the supplementary run, which measured all levels in one session, measured 8.14 tok/s, and that value is shown. The speed run itself measured 8 at once (per request) at 4.49 tok/s; the supplementary run, which measured all levels in one session, measured 4.4 tok/s, and that value is shown. The speed test ran with the settings the author's installer writes (Project Maya v1.1.22, setup choice 128K context, --tp 2 as its config file writes it, no drafting: the installer's own calibration measured 22.06 tok/s without the drafter and 18.37 with it), one slot, sampling temperature 1.0 and top_p 1.0 (the model card's and Maya's own default), seed 42, the request flag for no thinking. The headline 20.93 tok/s is the mean of the code prompt (22.91) and the prose prompt (18.945, shown as 18.9), medians of two runs each (spread 10.9 % and 13.0 %). The config says --tp 2, but the engine does not run tensor parallel on this model: its log says “tensor parallel 2 -> 1: the experts' 256-value blocks split n_ff 2304 no other way (each GPU its own layers)”. I report the setup as the author's setup writes it and changed nothing. The load monitor of my test harness listed, as the busiest other process, Maya's own engine during the speed and concurrency runs (about 1,000 % CPU: the CPU lane that serves the offloaded experts), so the load numbers of this setup do not mean foreign load there. Maya runs this 340 GB model as an expert tier over the 5090's VRAM, 96 GB of RAM and an NVMe SSD, so most of the model is offloaded: the engine log of the setup start counts about 1,970 of 15,360 experts in VRAM, and about 13,390 run on the CPU lane and are read in place from the model's files. Mia's Qwen3.8-27B in the same 32 GB class runs fully in VRAM, so the two sets of numbers are not built the same way. The server has 1 slot, so 2, 4 and 8 requests at once are served one after the other and the total stays near 19 to 20 tok/s. The “1 request at once” value is lower than the decode headline because that test counts the whole answer time, including the wait for the first token (about 9 to 11 s for its 1,280-token prompt). I measured this block three times (in the speed run and in two repeats); its two rounds differ strongly (12.2 and 16.5 tok/s in the last repeat; the spread stays above 15 %, with long first-token waits), so the 1-request value is only a rough guide. The page shows the last repeat. This is Project Maya v1.1.22, the current release when I set it up on 22 Oct 2026. Newer releases (v1.1.23 to v1.1.25, as of 23 Oct 2026) appeared afterwards; they are not measured on this page.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 speed2.py --base-url http://127.0.0.1:43521/v1 --label quality-v2-221022-maya-v1122-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/quality-v2-221022-maya-v1122-speed-r0/quality-v2-221022-maya-v1122-speed-r0/speed2 --contract $RUN/raw/quality-v2-221022-maya-v1122-speed-r0/quality-v2-221022-maya-v1122-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/maya-2110/tokenizer-hf/tokenizer.json --tokenizer-python $HOME/maya-v1122-src/.venv/bin/python --sampling {"temperature": 1.0, "top_p": 1.0} --sampling-source DeepSeek-V4.1-Flash-1017 model card (huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017, rev 1bae0e10, README.md:77-82 'Recommended sampling parameters' temperature 1.0, top_p 1.0, max_tokens >= 256K) and Project Maya's own default (maya.py:445 DS4_SAMPLING, written to maya-ds4.json by maya.py:3843). Same value for thinking on and off (the card names one set). --model deepseek-v4.1-flash --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 1 --server-slots-source mw00/project-maya @ffe2e5e5 (copies: ): the author's DeepSeek setup writes no --parallel (maya.py:3816-3845 write_config_ds4); --parallel defaults to 1 (maya.py:2611-2614 parallel_of '1', maya.py:3997-4000 help 'default 1', docs/SETTINGS.md:270 '--parallel N (1-8, default 1)'); the maya-ds4 engine announces 'READY <ctx> stop' without multi=<n> (src/ds4/ds4_main.cpp:337), so the server has 1 place (serve/server.py:423, :472 multi = 0; :1908 places = max(1, multi); :3389 /props total_slots = max(1, multi))
# Requests at once: supplementary run of 2026-10-22 (raw folder speed-r0-conc)
python3 speed2.py --base-url http://127.0.0.1:37395/v1 --label quality-v2-221022-maya-v1122-conc3-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/quality-v2-221022-maya-v1122-conc3-speed-r0/quality-v2-221022-maya-v1122-conc3-speed-r0/speed2 --contract $RUN/raw/quality-v2-221022-maya-v1122-conc3-speed-r0/quality-v2-221022-maya-v1122-conc3-speed-r0/speed2/speed-contract.json --parts conc --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/maya-2110/tokenizer-hf/tokenizer.json --tokenizer-python $HOME/maya-v1122-src/.venv/bin/python --sampling {"temperature": 1.0, "top_p": 1.0} --sampling-source DeepSeek-V4.1-Flash-1017 model card (huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017, rev 1bae0e10, README.md:77-82 'Recommended sampling parameters' temperature 1.0, top_p 1.0, max_tokens >= 256K) and Project Maya's own default (maya.py:445 DS4_SAMPLING, written to maya-ds4.json by maya.py:3843). Same value for thinking on and off (the card names one set). --model deepseek-v4.1-flash --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 1 --server-slots-source mw00/project-maya @ffe2e5e5 (copies: ): the author's DeepSeek setup writes no --parallel (maya.py:3816-3845 write_config_ds4); --parallel defaults to 1 (maya.py:2611-2614 parallel_of '1', maya.py:3997-4000 help 'default 1', docs/SETTINGS.md:270 '--parallel N (1-8, default 1)'); the maya-ds4 engine announces 'READY <ctx> stop' without multi=<n> (src/ds4/ds4_main.cpp:337), so the server has 1 place (serve/server.py:423, :472 multi = 0; :1908 places = max(1, multi); :3389 /props total_slots = max(1, multi)) --supplement-of quality-v2-221022-maya-v1122-speed-r0How well it answers
| Test | Result95 % margin | Right | Cut offcount as wrong | Raw files | |
|---|---|---|---|---|---|
| MMLU-Pro 400 fixed questions, letter only | 70.8 % ± 4.4 pts | 283 / 400 | 2 | summary · log · contract | command |
| GPQA Diamond 198 questions, letter only | 52.5 % ± 6.9 pts | 104 / 198 | 9 | answers · summary · log · contract | command |
| IFEval 200 fixed prompts, strict | 88.0 % ± 4.5 pts | 176 / 200 | 0 | answers · summary · log · contract | command |
| LiveCodeBench 40 newest tasks, the code runs | 60.0 % ± 14.5 pts | 24 / 40 | 0 | summary · log · contract | command |
| Long text 8K 4 tasks x 10 at 8192 tokens | 72.5 % ± 13.4 pts | 29 / 40 | 0 | log · contract · answers · summary | command |
| Long text 32K 4 tasks x 10 at 32768 tokens | 52.5 % ± 14.8 pts | 21 / 40 | 1 | log · contract · answers · summary | command |
| GPQA 30, thinking on 30 questions, limit 16,384 tokens | 80.0 % ± 13.9 pts median 2,092 tokens, 86.4 s per question | 24 / 30 | 5 | log · contract · answers · summary | command |
Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-23.
Note from the run: MMLU-Pro and GPQA Diamond ran with an answer limit of 1,024 tokens. In some answers Maya explains before it gives the letter: in my first run with the usual 48-token limit, 44 of 400 MMLU-Pro answers and 47 of 198 GPQA answers were cut off and counted as wrong. I ran both tests again with 1,024 tokens (same questions, seed 42, same sampling and setup). This page shows only the 1,024-token values and never mixes them with the 48-token run. Even at 1,024 tokens, 2 of 400 MMLU-Pro answers and 9 of 198 GPQA answers were cut off and count as wrong. 1 of the 80 long-text answers (32K) and 5 of 30 GPQA answers with thinking (limit 16,384 tokens) were cut off too. LiveCodeBench and the long-text tests were finished in a second part of the run: items that were already done before a broken-pipe error in the first part were taken over (same card, lists and sampling), the rest was run again. GPQA Diamond ran on the public mirror fingertap/GPQA-Diamond (the original is gated). During the quality tests the load monitor listed the kernel's memory manager (kswapd0) as the busiest other process in almost all samples, usually near 20 % of one core, and with 6 single samples of 50 to 100 % of one core, from the harness's server-ready check (a short Python call), the code of a LiveCodeBench solution that was being run, and a Python worker. These are my own tests.
Exact commands we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
bash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts mmlupro --job quality-v2-221022-maya-v1122-mcq1024bash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts gpqa --job quality-v2-221022-maya-v1122-mcq1024bash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts ifeval --job quality-v2-221022-maya-v1122-qbash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts lcb --job quality-v2-221022-maya-v1122-qbash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts ruler --job quality-v2-221022-maya-v1122-qbash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts ruler --job quality-v2-221022-maya-v1122-qbash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts think --job quality-v2-221022-maya-v1122-qHow much context fits
| Context size | Prompttokens | Filled | First token | Prefilltok/s, as the server reports it | Decodetok/s | Peak memoryGiB | Result |
|---|---|---|---|---|---|---|---|
| 4K 4,096 tokens | 3,891 | 95.0 % | — | 314 | 10.4 | 30.91 | answered (stop) |
| 8K 8,192 tokens | 7,782 | 95.0 % | — | 506 | 12.7 | 30.91 | answered (stop) |
| 16K 16,384 tokens | 15,564 | 95.0 % | — | 804 | 15.5 | 30.91 | answered (stop) |
| 32K 32,768 tokens | 31,130 | 95.0 % | — | 1,186 | 15.7 | 30.92 | answered (stop) |
| 64K 65,536 tokens | 62,259 | 95.0 % | — | 1,557 | 13.6 | 30.92 | answered (stop) |
| 128K 131,072 tokens | 124,518 | 95.0 % | — | 1,823 | 14.7 | 30.91 | answered (stop) |
Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. Measured 2026-10-22. Raw files: result of every step · log · contract.
Note from the run: The largest size tested is 131,072 tokens, the context of this setup (the 128K choice in the author's installer, which offers larger contexts too), so it is the largest size tested, not necessarily the largest the engine could load. Every size from 4K to 128K loaded and answered. The memory number is the GPU memory in use on the 5090; most of the model sits in RAM and on the NVMe SSD, which this number does not show. The decode value is the speed at the largest size.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
bash tools/quality2/measure_all.sh tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json --parts fit --job quality-v2-221022-maya-v1122Also made this possible
- Model
DeepSeekHugging Face @deepseek_ai GitHub Website - Maya-S GGUF weightsDDeepSeek-V4.1-Flash-1017-Maya-GGUFHugging Face
Run it yourself
What you need and how to start it, step by step. The results below come from exactly this setup.
What you need
- Hardware
- 1x NVIDIA GeForce RTX 5090 32 GB, 96 GB RAM, 4 TB NVMe SSD (expert tier: most of the 340 GB model is offloaded to RAM and NVMe)
- Model
- peasantsmith/DeepSeek-V4.1-Flash-1017-Maya-GGUF Maya-S (Project Maya's own quant, 748B DeepSeek-V4.1-Flash-1017; 6 shards, sha256 equal to the pinned table of v1.1.22)
- Maya-S GGUF weights
- huggingface.co/peasantsmith/DeepSeek-V4.1-Flash-1017-Maya-GGUF
- Engine
- maya-ds4 v1.1.22 on one RTX 5090 with 96 GB RAM and NVMe (expert tier, the author's installer settings) — the program that loads the model and answers requests · github.com/mw00/project-mayaBuild and settings we used: Project Maya maya-ds4 v1.1.22 (mw00/project-maya @ffe2e5e5), CUDA 13.1-User-Toolkit-Build, sm_120
Steps
Get the engine and the model files from the list above.
Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8080.
.venv/bin/python maya.py \ --config maya-ds4.json \ --port 8080 \ --plain (engine build/maya-ds4 \ --pack .../Maya-S/maya-ds4 \ --max-context 131072 \ --tp 2 \ --dspark none, Port 8080; config written by ./setup.sh); via box-6-maya-ds4-v1122.sh startThe file names are the ones we used. Run the command in the folder that holds them, or put the full path in front of each name.
What the parts mean
--port 8080the network port the server listens on
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the start text exactly as our run recorded it, with the engine build in front, so anyone can repeat the result.
Project Maya maya-ds4 v1.1.22 (mw00/project-maya @ffe2e5e5), CUDA 13.1-User-Toolkit-Build, sm_120; .venv/bin/python maya.py --config maya-ds4.json --port 8080 --plain (engine build/maya-ds4 --pack .../Maya-S/maya-ds4 --max-context 131072 --tp 2 --dspark none, Port 8080; config written by ./setup.sh); via $HOME/maya-2110/box-6-maya-ds4-v1122.sh startContext and sampling: 131,072 tokens (the 128K choice in the author's installer; the installer's default is 64K), one slot, seed 42, requests ask for no prompt cache. Sampling as the model card and Maya's own default give it: temperature 1.0, top_p 1.0 (one set for thinking on and off). Reasoning effort high (the author's default). Tensor parallel: the config says --tp 2 and the engine falls back to 1 (see the speed note). Speed with thinking off; the build test with thinking on. Answer limit for the build test 262,144 tokens (the model card), lowered by my test gate to the room left in the context. Agent for the build test: DeepSeek Harness 0.2.1-rc.2. The vision tower and the drafter files are in the pack (installer default); the drafter is switched off (see the speed note).. Hardware: 1x NVIDIA GeForce RTX 5090 32 GB, 96 GB RAM, 4 TB NVMe SSD (expert tier: most of the 340 GB model is offloaded to RAM and NVMe). All raw files of this setup: raw/v2/deepseek-v41-flash-5090-maya-s-mayads4.
Mia's own benchmark, run unchanged
This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section on this page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.
| Test | Result95 % margin | Right | Cut offcount as wrong | Raw files | |
|---|---|---|---|---|---|
| Quality QA 150 questions | 90.0 % ± 4.8 pts | 135 / 150 | 2 | summary | command |
| Quality MMLU 285 questions | 91.6 % ± 3.3 pts | 261 / 285 | 0 | summary | command |
| Tool Eval Quick 15 scenarios, tool use | 87 / 100 | 12 / 15 passed | — | tool eval result · tool eval report · tool eval command · summary | command |
| Decode, 1 request at once first token 45 ms | 31.6 tok/s | — | — | decode result · summary | command |
| Prefill at 4K first token 5.1 s | 811.1 tok/s | — | — | prefill result · summary | command |
| Prefill at 8K first token 6.4 s | 1294.8 tok/s | — | — | prefill result · summary | command |
| Prefill at 16K first token 8.7 s | 1884.9 tok/s | — | — | prefill result · summary | command |
| Prefill at 32K first token 14.8 s | 2221.4 tok/s | — | — | prefill result · summary | command |
Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-22. Version 2.50.0, commit 21a3511.
QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).
MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).
Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.
Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.
Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.
Note from the run: sparkDash runs its own tests (its own QA and MMLU questions at temperature 0, thinking off, a cut-off answer counts as wrong; 2 of the QA answers were cut off, none of the MMLU answers), so its decode and prefill values are not comparable with Speed v2 above. The sparkDash client ran on another machine and was never part of the measured group.
Exact commands we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 tools/quality2/sparkdash_client.py tools/quality2/cards/ds1017-maya-s-mayads4-1x-128k-v1122.json <box> --client <client machine> --job maya-v1122-sparkdash-2210 --out $RUNtool-eval-bench --base-url http://127.0.0.1:<port> --model deepseek-v4.1-flash --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/maya-v1122-sparkdash-2210-sparkdash-r0/tooleval/result.json --no-live