neroued's NInfer: Qwen3.8-27B (NVFP4 + MTP)

What it builds as an agent

Task: Airship dock (3D scene, one HTML file). One attempt, no help. Counted: the final file counts.

Design and build the most impressive, creative, elaborate and detailed steampunk airship dock voxel world you are capable of. Theme: a steampunk airship dock. Everything else (style, story, extra details) is your choice.

How you work:
- You work alone in the folder /work. Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes. Spend the time where it shows in the result.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh index.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- You may create your own assets locally with these tools (textures, sky, sounds, music) and embed them in the file as data URIs. Do not download anything.
- Keep a working /work/index.html at all times: write a first working version early, then improve it step by step. If you are stopped, the last version that loaded without errors counts.
- Check your work with `shot.sh` as often as you like, fix every error, and keep improving until you are satisfied. Then stop.

Deliverable: ONE self-contained file /work/index.html that opens and runs in Chrome.
- JavaScript libraries: only these, through an import map with these exact URLs (they work offline in the test browser). Use whatever helps, including all Three.js addons (post-processing, sky, water, noise and so on):
  - "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js"
  - "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/"
  - "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js"
  - "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js"
  - "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js"
- No other external resources. Build every object from cubes (voxels); InstancedMesh is allowed.
- The page fills the whole window and resizes with it. All visible text is in English.
- After the scene is built, set `window.scene` to the Three.js scene and `window.ready = true`.
- An automatic slow cinematic camera tour starts on load. Mouse input switches to OrbitControls (drag to rotate, scroll to zoom).
- Sound, if any, starts only after the first click (browsers block autoplay).
- A small label in the top-left corner: "Steampunk Airship Dock".

Minimum content (go far beyond it):
1. A tall iron docking tower with platforms, stairs or ladders, and a mooring arm at the top.
2. At least two airships, each with a balloon envelope, a gondola and propellers.
3. Big brass gears on the tower or the buildings.
4. Pipes and at least three chimneys that emit rising steam particles.
5. At least one crane and a stack of cargo crates.
6. A ground area with a workshop building and a small rail track.
7. Propellers and gears rotate, steam rises and fades, and one airship slowly approaches the tower, docks, waits and leaves again, in a loop.
8. Warm sunset lighting with shadows.

The browser check

• page loads without page error1 / 1
• window.ready === true within 30 s1 / 1
• no console errors (whole session)1 / 1
• frame is not blank1 / 1
• frame rate in software GL (rAF over 5 s)0 / 1
• required objects (voxel mass, >=3 steam emitters, >=2 airships, label)1 / 2
• motion: pixel diff 0.5, >=3 rotating nodes 0.5, large group translating 12 / 2
• OrbitControls: drag and wheel change the picture beyond the animation baseline0 / 1

Measured 2026-10-24. Raw files: check · run info · command · final choice · contract · sampling · gate stats · screenshot · clip.mp4 · web.mp4 · web-poster.jpg.

The run's own headless check browser, which renders the page the model built, used up to 1490 % CPU. The load monitor counted it as another process. Browser check 7.0 out of 10 (airship-dock task): the page loads without a page error, is ready after 0.01 s, has no console errors, is not blank, and the scene moves (the motion check gave its 2 points). The 3 lost points are the frame rate (5.95 frames per second in software rendering without a GPU, 10 give half a point and 20 the full point), the objects check (1 steam source found where 3 are needed for the full 2 points) and the orbit-controls test (drag and wheel changed the picture less than the check needs). The agent (Qwen Code 0.25.2) finished by itself after 56.5 minutes, one try, no help; the “finish now” message (from 90 minutes) was not needed.7 versions of the page were seen on the way, 6 of them loaded without errors, and the last one counts. 69 requests, no error, no request blocked (7,666,202 prompt tokens, 207,007 answer tokens). The limit per answer was 262,144 tokens, the value on the Qwen3.8-27B model card (README lines 507 to 510); my test proxy lowered it to the room left in the 240,000-token context in 29 of the 69 requests. The proxy estimates the size of a prompt without the server's tokenizer, and once the estimate was too high (up to 1.63 times the real size, see gate-stats.json); it then cut one answer limit more than needed and the harness asked again. That is a fault of my test tool, not of the setup. Thinking was on with the model card's sampling (temperature 1.0, top_p 0.95, top_k 20), the context was 240,000 tokens, 2 lanes.

run_showcase.sh --harness qwen-code --task K1 --upstream http://127.0.0.1:32811/v1 --model qwen3.8-27b --sampling-file $RUN/sampling-show.json --out $RUN/raw/nacht2410-show-r5/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 240000

Extra build: Voxel world (not part of the score)

The 32 GB class is ranked on the airship-dock task. This voxel-world run used a different prompt (the one the 128 GB+ classes use), so it is shown as an extra and does not count toward the score or the ranking.

Task: Voxel world (3D scene, one HTML file). One attempt, no help. Which file is used: the final file counts.

Design and create the most impressive, creative, elaborate and detailed
voxel art world you are capable of. The theme and style are ENTIRELY your
choice — pick whatever world you can execute best and push it as far as
you can: scale, variety, detail, color, animation, atmosphere, life.
Surprise me.

Hard requirements:
- Everything ends up in ONE single self-contained HTML file (game.html in
  the current working directory) that opens and runs directly in Chrome.
  Use WebGL and whatever libraries you like, but embed everything —
  no internet access at view time, no external files.
- The scene must present itself: include an automatic slow cinematic
  camera tour (orbiting the world, occasionally zooming in on highlights)
  that starts on load, plus mouse controls (orbit/zoom) for manual viewing.
- All user-facing text (UI, HUD, titles, descriptions, hints) must be in
  ENGLISH, regardless of the language of any project files you encounter.

When finished, verify your HTML actually loads and renders without
console errors, then state DONE with the absolute path of game.html.
Do not ask questions.

How you work (same for every model):
- You work alone in the folder /work (this is the current working directory). Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh game.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- Keep a working /work/game.html at all times. If you are stopped, the last version that loaded without errors counts.
- JavaScript libraries available offline in the test browser, through an import map with these exact URLs: "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js", "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/", "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js", "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js", "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js". No other external resources.
- After the scene is built, set `window.scene` to the Three.js scene (if you use Three.js) and `window.ready = true`.

The browser check

• page loads without page error1 / 1
• window.ready === true within 30 s1 / 1
• no console errors (whole session)1 / 1
• frame is not blank1 / 1
• frame rate in software GL (rAF over 5 s)1 / 1
• world: scene size 0.5 (parts >= 80 or >= 20000 triangles; canvas fallback), visible text English if any 0.51 / 1
• automatic camera tour: the view changes on its own in two successive ~5 s windows (1 each)0 / 2
• animation in the world (nodes / instances / particles moving besides the camera)1 / 1
• OrbitControls: drag and wheel change the picture beyond the animation baseline1 / 1

Measured 2026-10-24. Raw files: check · run info · command · final choice · contract · sampling · gate stats · screenshot · clip.mp4.

The run's own headless check browser, which renders the page the model built, used up to 1506 % CPU. The load monitor counted it as another process. Extra build, not part of the score. This run used the voxel-world prompt (the one the 128 GB and larger classes use), not the airship-dock prompt this page is ranked on. Same box, same server settings, same agent as the airship-dock run above; only the prompt differs. Browser check 8.0 out of 10: the page loads without a page error, is ready after 0.27 s, has no console errors, is not blank, shows a world (532 parts, 56,402 triangles, English text), runs at 38.67 frames per second in software rendering, animates (79 animated nodes) and answers to mouse drag and wheel. The 2 lost points are the automatic camera tour: the view changed too little in the two windows of the check (4.8 % and 4.2 % of the picture changed, the check needs 5 %). The agent finished by itself after 64.2 minutes, one try, no help; the “finish now” message (from 90 minutes) was not needed. 24 versions of the page were seen on the way, 19 loaded without errors, and the last one counts. 72 requests, no error, no request blocked (10,874,828 prompt tokens, 182,070 answer tokens); my test proxy lowered the answer limit (262,144 tokens, the model card) to the room left in the 240,000-token context in 50 of the 72 requests. Thinking was on with the model card's sampling (temperature 1.0, top_p 0.95, top_k 20), 2 lanes.

run_showcase.sh --harness qwen-code --task VX --upstream http://127.0.0.1:40477/v1 --model qwen3.8-27b --sampling-file $RUN/sampling-show.json --out $RUN/raw/nacht2410-show-r0/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 240000

How fast it runs

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; requests at once (later supplementary run of 2026-10-24, same setup): C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-10-24. Raw files: speed result · speed contract · log · long answer · speed result · speed contract · log · long answer.

Requests at once: the server has 8 slots (the author's command line sets 2 and his README allows 1 to 8, README.md:72 and :251; I took the largest that starts with the full context of 240,000, docs/serving.md:913 and :941). The speed test ran with the author's setup (Neroued/ninfer, commit 64d10fb of 16 Oct 2026, built here from source with CUDA 13.3): NInfer, an engine written from scratch in C++ and CUDA for one RTX 5090, with the author's NVFP4 file of Qwen3.8-27B (neroued/Qwen3.8-27B-nvfp4-NInfer, revision a107ba1b5b06), an FP8 KV cache, MTP with 3 draft tokens and the lm-head draft, a context of 240,000 tokens and prefix reuse on. The start line is the one in his README (lines 69 to 77) with only host and port added; it sets 2 lanes (--max-concurrency 2). Decode, prefill and the speed headline come from this run with 2 lanes. Sampling is the engine's preset for the model, which is the model card's: temperature 0.7, top_p 0.8, top_k 20, presence penalty 1.5 with thinking off; seed 42; no prompt cache in the requests (every request carries its own nonce). The headline 159.56 tok/s is the mean of the code prompt (168.98) and the prose prompt (150.15), medians of two runs each (spread 7.4 % and 0.9 %). Requests at once: his command line sets 2 lanes, and his README says the engine takes 1 to 8 lanes (README.md:251). As my test method asks, I took the largest setting that starts with the full context of 240,000 tokens, which is 8 lanes (--max-concurrency 8, everything else in the line unchanged, --device-state-slots stays 2), and measured C1, C2, C4 and C8 in a separate run of the speed test on the same box on the same day (raw folder speed-r0-conc). The 2-lane run measured these levels too (149.8, 275.1, 269.0 and 280.9 tok/s in total for 1, 2, 4 and 8 at once); those values are in the raw files of the main run and are not the ones shown. The requests-at-once values are therefore for more lanes than the author's own line uses, and the page names it here.

python3 speed2.py --base-url http://127.0.0.1:32905/v1 --label nacht2410-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/nacht2410-speed-r0/nacht2410-speed-r0/speed2 --contract $RUN/raw/nacht2410-speed-r0/nacht2410-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/models-2410/kaitchup_Qwen3.8-27B-SLQ-EAR98/tokenizer.json --tokenizer-python $HOME/hfvenv/bin/python --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source https://huggingface.co/Qwen/Qwen3.8-27B README.md lines 252 (Thinking Mode) and 253 (Instruct/non-thinking mode), local copy not published line 252-253 (not published) --model qwen3.8-27b --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 2 --server-slots-source Neroued/ninfer@64d10fb README.md:72 (--max-concurrency 2) + :74 (--device-state-slots 2); docs/serving.md:21-23. The engine accepts 1..8 lanes (README.md:251); the author's line uses 2, so C4/C8 queue behind 2 lanes (bounded FIFO ingress). Issue #414: 4 identical concurrent requests can fail.

# Requests at once: supplementary run of 2026-10-24 (raw folder speed-r0-conc)
python3 speed2.py --base-url http://127.0.0.1:34807/v1 --label nacht2410-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/nacht2410-speed-r0/nacht2410-speed-r0/speed2 --contract $RUN/raw/nacht2410-speed-r0/nacht2410-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/models-2410/kaitchup_Qwen3.8-27B-SLQ-EAR98/tokenizer.json --tokenizer-python $HOME/hfvenv/bin/python --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source https://huggingface.co/Qwen/Qwen3.8-27B README.md lines 252 (Thinking Mode) and 253 (Instruct/non-thinking mode), local copy not published line 252-253 (not published) --model qwen3.8-27b --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 8 --server-slots-source Neroued/ninfer@64d10fb README.md:251 (one to eight resident execution lanes) + docs/serving.md:913,941; largest --max-concurrency that starts with --max-context 240000 --kv-capacity 240000 (own start test, Vorschritt lanes8-pre.sh, log queue-2410/logs/nl1.pre.log). The author's own line uses 2 (README.md:72). Opening note: this is more concurrency than the author's line.

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-24.

Note from the run: MMLU-Pro and GPQA Diamond ran with an answer limit of 48 tokens (my usual limit for these two tests, the answer is one letter); none of the answers of these two tests was cut off. 2 of 40 LiveCodeBench answers were cut off at their limit of 32,768 tokens and 7 of 30 GPQA answers with thinking at the limit of 16,384 tokens; these count as wrong. None of the long-text answers was cut off. The questions of each test ran 2 at a time, the lane count of the author's line (same questions, limits and seed 42 for every question). Sampling: with thinking off temperature 0.7, top_p 0.8, top_k 20, presence penalty 1.5; with thinking on (GPQA 30) temperature 1.0, top_p 0.95, top_k 20 (the model card). These are my own tests, and no other run of this setup is mixed into the numbers on this page.

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts mmlupro --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts gpqa --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts ifeval --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts lcb --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts ruler --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts ruler --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts think --job nacht2410

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. Measured 2026-10-24. Raw files: result of every step · log · contract.

Note from the run: The context is the author's own 240,000 tokens, which is the largest size tested here; the sizes from 4K to 128K came before it. For each size the server was started again with that context (--max-context and --kv-capacity), one prompt that fills about 95 % of the size was sent, and the model answered at every size. The memory number is the GPU memory in use on the 5090 (28.94 GiB at 240,000 tokens, with a prompt of 228,000 tokens). The decode value is the speed at the largest size (97.85 tok/s).

bash tools/quality2/measure_all.sh tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json --parts fit --job nacht2410

Also made this possible

• ModelQwenHugging Face @Alibaba_Qwen GitHub Website
• Model file (NVFP4, revision a107ba1b5b06)nerouedHugging Face GitHub

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5090 32 GB, one GPU (NInfer is a single-GPU engine).

Model

neroued/Qwen3.8-27B-nvfp4-NInfer (qwen3_8_27b_nvfp4.ninfer, NVFP4/FP8 mixed, base unsloth/Qwen3.8-27B-NVFP4)

Model file (NVFP4, revision a107ba1b5b06)

huggingface.co/neroued/Qwen3.8-27B-nvfp4-NInfer

Engine

NInfer engine for one RTX 5090, NVFP4 weights, MTP with 3 draft tokens, FP8 KV cache, 240,000 tokens of context — the program that loads the model and answers requestsBuild and settings we used: NInfer 64d10fb (source build, CUDA 13.3, sm_120a), FP8 KV, MTP3 + lm-head-draft, 2 lanes, prefix reuse on

Steps

1. Get the model files from the list above. The list has no link for the engine (NInfer engine for one RTX 5090, NVFP4 weights, MTP with 3 draft tokens, FP8 KV cache, 240,000 tokens of context).

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   ninfer-serve qwen3_8_27b_nvfp4.ninfer \
     --host 127.0.0.1 \
     --port 8888 \
     --max-context 240000 \
     --kv-capacity 240000 \
     --max-concurrency 2 \
     --kv-dtype fp8 \
     --device-state-slots 2 \
     --spec mtp \
     --draft-tokens 3 \
     --lm-head-draft \
     --preserve-thinking

   What the parts mean

   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on

NInfer 64d10fb (source build, CUDA 13.3, sm_120a), FP8 KV, MTP3 + lm-head-draft, 2 lanes, prefix reuse on; ninfer-serve qwen3_8_27b_nvfp4.ninfer --host 127.0.0.1 --port 8888 --max-context 240000 --kv-capacity 240000 --max-concurrency 2 --kv-dtype fp8 --device-state-slots 2 --spec mtp --draft-tokens 3 --lm-head-draft --preserve-thinking

Mia's own benchmark, run unchanged

This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section on this page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.

Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-24. Version 2.50.0, commit 21a3511.

QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.

Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.

Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.

Note from the run: sparkDash runs its own tests (its own QA and MMLU questions at temperature 0, thinking off, a cut-off answer counts as wrong; none of the QA answers and 19 of the 285 MMLU answers were cut off), so its decode and prefill values are not comparable with Speed v2 above: its decode test uses a structured prompt, Speed v2 a code and a prose prompt. The sparkDash client ran on another machine and was never part of the measured group.

python3 tools/quality2/sparkdash_client.py tools/quality2/cards/qwen38-27b-nvfp4-ninfer-mtp3-5090.json <box> --client <client machine> --job nacht2410 --out $RUN

tool-eval-bench --base-url http://127.0.0.1:<port> --model qwen3.8-27b --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/nacht2410-sparkdash-r6/tooleval/result.json --no-live