The Kaitchup's Qwen3.8-27B-SLQ-EAR98 on vLLM 0.31.0

What it builds as an agent

Task: Airship dock (3D scene, one HTML file). One attempt, no help. Counted: the final file counts.

Design and build the most impressive, creative, elaborate and detailed steampunk airship dock voxel world you are capable of. Theme: a steampunk airship dock. Everything else (style, story, extra details) is your choice.

How you work:
- You work alone in the folder /work. Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes. Spend the time where it shows in the result.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh index.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- You may create your own assets locally with these tools (textures, sky, sounds, music) and embed them in the file as data URIs. Do not download anything.
- Keep a working /work/index.html at all times: write a first working version early, then improve it step by step. If you are stopped, the last version that loaded without errors counts.
- Check your work with `shot.sh` as often as you like, fix every error, and keep improving until you are satisfied. Then stop.

Deliverable: ONE self-contained file /work/index.html that opens and runs in Chrome.
- JavaScript libraries: only these, through an import map with these exact URLs (they work offline in the test browser). Use whatever helps, including all Three.js addons (post-processing, sky, water, noise and so on):
  - "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js"
  - "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/"
  - "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js"
  - "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js"
  - "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js"
- No other external resources. Build every object from cubes (voxels); InstancedMesh is allowed.
- The page fills the whole window and resizes with it. All visible text is in English.
- After the scene is built, set `window.scene` to the Three.js scene and `window.ready = true`.
- An automatic slow cinematic camera tour starts on load. Mouse input switches to OrbitControls (drag to rotate, scroll to zoom).
- Sound, if any, starts only after the first click (browsers block autoplay).
- A small label in the top-left corner: "Steampunk Airship Dock".

Minimum content (go far beyond it):
1. A tall iron docking tower with platforms, stairs or ladders, and a mooring arm at the top.
2. At least two airships, each with a balloon envelope, a gondola and propellers.
3. Big brass gears on the tower or the buildings.
4. Pipes and at least three chimneys that emit rising steam particles.
5. At least one crane and a stack of cargo crates.
6. A ground area with a workshop building and a small rail track.
7. Propellers and gears rotate, steam rises and fades, and one airship slowly approaches the tower, docks, waits and leaves again, in a loop.
8. Warm sunset lighting with shadows.

The browser check

• page loads without page error1 / 1
• window.ready === true within 30 s1 / 1
• no console errors (whole session)1 / 1
• frame is not blank1 / 1
• frame rate in software GL (rAF over 5 s)0 / 1
• required objects (voxel mass, >=3 steam emitters, >=2 airships, label)2 / 2
• motion: pixel diff 0.5, >=3 rotating nodes 0.5, large group translating 12 / 2
• OrbitControls: drag and wheel change the picture beyond the animation baseline0 / 1

Measured 2026-10-25. Raw files: check · run info · command · final choice · contract · sampling · gate stats · gate log · screenshot · clip.mp4 · web.mp4 · web-poster.jpg.

The run's own headless check browser, which renders the page the model built, used up to 1509 % CPU. The load monitor counted it as another process. Browser check 8 out of 10 (airship-dock task): the page loads without a page error, is ready after 0.01 s, has no console errors, is not blank, shows the required objects (2 airships, 4 steam sources, a voxel mass of about 7,350 voxels, and the label 'Steampunk Airship Dock' visible in the top left), and animates and moves (15 rotating nodes, large groups travelling). The 2 lost points are the frame rate (7.1 frames per second in software rendering without a GPU, 10 give half a point and 20 the full point) and the orbit-controls test (the mouse drag and the wheel changed the picture less than the check needs). The build test ran twice. In the first try, at the context of 74,752 tokens, the model's thinking used up the whole answer limit of one request (41,302 tokens, the room the context left), so it never wrote a page: 0 points, and that try is not shown on this page. For the second try I started the server again with more room for one long request: context 126,976 tokens, --max-num-seqs 1 and --gpu-memory-utilization 0.95 (cache still bf16). The second try is the one that counts and the one on this page. Speed, quality and sparkDash were not repeated; they stay at the first setting. The agent (Qwen Code 0.25.2) took 34.0 minutes in the second try, no help; it ended by itself, so the 'finish now' message (from 90 minutes) was not needed. 34 requests, no error, none blocked (1,584,713 prompt tokens, 105,854 answer tokens, largest prompt 102,946 of 126,976 tokens). 9 versions of the page were seen on the way, 8 loaded without errors, and the last one counts. Thinking was on with the base card's thinking sampling (temperature 1.0, top_p 0.95, top_k 20). The limit per answer was 262,144 tokens (the Qwen3.8-27B card's figure), cut for 5 requests to the free context. What it built, according to its own report: a voxel iron docking tower with two platforms, three airships (one docking, one leaving, one circling), spinning brass gears, copper pipes, steam from emitters, a swinging crane, a brick workshop with a sign and a rail cart. The load monitor saw up to 1,509 % CPU; that was the test's own headless check browser.

run_showcase.sh --harness qwen-code --task K1 --upstream http://127.0.0.1:34665/v1 --model kaitchup/Qwen3.8-27B-SLQ-EAR98 --sampling-file $RUN/sampling-show.json --out $RUN/raw/nacht2410-show-r0/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 126976

Extra build: Voxel world (not part of the score)

The 32 GB class is ranked on the airship-dock task. This voxel-world run used a different prompt (the one the 128 GB+ classes use), so it is shown as an extra and does not count toward the score or the ranking.

Task: Voxel world (3D scene, one HTML file). One attempt, no help. Which file is used: the final file counts.

Design and create the most impressive, creative, elaborate and detailed
voxel art world you are capable of. The theme and style are ENTIRELY your
choice — pick whatever world you can execute best and push it as far as
you can: scale, variety, detail, color, animation, atmosphere, life.
Surprise me.

Hard requirements:
- Everything ends up in ONE single self-contained HTML file (game.html in
  the current working directory) that opens and runs directly in Chrome.
  Use WebGL and whatever libraries you like, but embed everything —
  no internet access at view time, no external files.
- The scene must present itself: include an automatic slow cinematic
  camera tour (orbiting the world, occasionally zooming in on highlights)
  that starts on load, plus mouse controls (orbit/zoom) for manual viewing.
- All user-facing text (UI, HUD, titles, descriptions, hints) must be in
  ENGLISH, regardless of the language of any project files you encounter.

When finished, verify your HTML actually loads and renders without
console errors, then state DONE with the absolute path of game.html.
Do not ask questions.

How you work (same for every model):
- You work alone in the folder /work (this is the current working directory). Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh game.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- Keep a working /work/game.html at all times. If you are stopped, the last version that loaded without errors counts.
- JavaScript libraries available offline in the test browser, through an import map with these exact URLs: "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js", "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/", "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js", "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js", "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js". No other external resources.
- After the scene is built, set `window.scene` to the Three.js scene (if you use Three.js) and `window.ready = true`.

The browser check

• page loads without page error1 / 1
• window.ready === true within 30 s1 / 1
• no console errors (whole session)1 / 1
• frame is not blank1 / 1
• frame rate in software GL (rAF over 5 s)0 / 1
• world: scene size 0.5 (parts >= 80 or >= 20000 triangles; canvas fallback), visible text English if any 0.51 / 1
• automatic camera tour: the view changes on its own in two successive ~5 s windows (1 each)2 / 2
• animation in the world (nodes / instances / particles moving besides the camera)1 / 1
• OrbitControls: drag and wheel change the picture beyond the animation baseline0.5 / 1

Measured 2026-10-25. Raw files: check · run info · command · final choice · contract · sampling · gate stats · gate log · screenshot · clip.mp4.

The run's own headless check browser, which renders the page the model built, used up to 1572 % CPU. The load monitor counted it as another process. Extra build, not part of the score. This run used the voxel-world prompt (the one the 128 GB and larger classes use), not the airship-dock prompt this page is ranked on. Same card, same server settings as the dock build (context 126,976, --max-num-seqs 1, --gpu-memory-utilization 0.95), same agent (Qwen Code 0.25.2), same limits and sampling; only the prompt differs. My rating of this extra world: 4.5 of 10 (it does not count in the score). The agent finished by itself after 32.9 minutes, one try, no help. 10 requests, no error (573,920 prompt tokens, 106,504 answer tokens, largest prompt 103,924 of 126,976 tokens); one request, close to the end of the context, was given a limit of only 16 tokens by the test gate. One version of the page was seen on the way and it loaded without errors. Browser check 8.5 out of 10: the frame rate lost its point (1.69 frames per second in software rendering) and the orbit-controls test gave half a point. What it built: LUMINARA, 'a floating voxel isle at dusk': 45,133 parts, 543,632 triangles, 77 animated nodes, an automatic camera tour, English text.

run_showcase.sh --harness qwen-code --task VX --upstream http://127.0.0.1:38331/v1 --model kaitchup/Qwen3.8-27B-SLQ-EAR98 --sampling-file $RUN/sampling-show.json --out $RUN/raw/nacht2410-show-r0/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 126976

How fast it runs

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-10-25. Raw files: speed result · speed contract · log · long answer.

Requests at once: the recipe's slot count is 1 (at the full context the KV cache holds about 1.01 requests of full length), but the server ran with --max-num-seqs 8 (my addition), so C1, C2, C4 and C8 are real parallel runs of short requests; long prompts at the same time would have to queue. The speed test ran the author's serve command (kaitchup/Qwen3.8-27B-SLQ-EAR98, README 'Instructions: vLLM') on vLLM 0.31.0 with humming-kernels 0.1.16, bf16 cache, no speculation (the author did not include speculative decoding in his validation; the card's original MTP weights stay in the model but were not used). The author tested this model on an RTX PRO 6000 Blackwell with 96 GB, and his serve command sets a context of 262,144 tokens. That does not fit this 32 GB card: after loading the 23.7 GB of weights, vLLM reports 73,728 tokens as the most it can hold. So speed, quality, the memory test and sparkDash ran at a context of 74,752 tokens, the largest size that loaded. His command also did not start on 32 GB at all: by default vLLM's start-up check sets aside a minimum cache for 512 sequences (12.25 GiB, no matter how long the context is), so I added one flag, --max-num-seqs 8. The cache is bf16 (his card names no cache type). Nothing else in his command was changed. Requests at once: the author's command sets no limit for parallel requests. At the context of 74,752 tokens the cache holds about one request of full length (the vLLM server log says 'Maximum concurrency ... 1.01x'), which is the slot count of 1 shown on this page. The server itself ran with --max-num-seqs 8, so C1, C2, C4 and C8 are real parallel runs of 512-token answers on short prompts; long prompts at the same time would have to queue. Sampling is the non-thinking sampling of the base Qwen3.8-27B card (temperature 0.7, top_p 0.8, top_k 20, presence penalty 1.5), because the Kaitchup card gives none; thinking off, seed 42, no prompt cache. The headline 67.15 tok/s is the mean of the code prompt (67.20) and the prose prompt (67.11), medians of two runs each (spread 0.15 % and 0.17 %). 500 words take about 10.7 s (500 divided by 46.91 words per second). There is no draft acceptance number, because nothing was drafted.

python3 speed2.py --base-url http://127.0.0.1:45497/v1 --label nacht2410-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/nacht2410-speed-r0/nacht2410-speed-r0/speed2 --contract $RUN/raw/nacht2410-speed-r0/nacht2410-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/models-2410/kaitchup_Qwen3.8-27B-SLQ-EAR98/tokenizer.json --tokenizer-python $HOME/hfvenv/bin/python --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source https://huggingface.co/Qwen/Qwen3.8-27B README.md lines 252 (Thinking Mode) and 253 (Instruct/non-thinking mode), local copy not published line 252-253 (not published) --model kaitchup/Qwen3.8-27B-SLQ-EAR98 --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 1 --server-slots-source vLLM 0.31.0 server log line 'Maximum concurrency for 74,752 tokens per request: 1.01x' ($HOME/queue-2410/logs/vllm-kaitchup-server.log) = KV-limited concurrency at the served context, floor, capped at 8; the author's line sets no --max-num-seqs (vLLM default applies, not a slot recipe); extra args: --max-num-seqs 8

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-25.

Note from the run: Quality ran on the first setting (context 74,752, --max-num-seqs 8), one question at a time. MMLU-Pro and GPQA Diamond ran with a short answer limit (the answer is one letter); IFEval and LiveCodeBench with longer limits. No answer was cut off in the five instruct tests (0). In LiveCodeBench 3 of the 40 answers contained no code and count as wrong. GPQA with thinking has a limit of 16,384 tokens; 8 of its 30 answers hit it and count as wrong. GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 questions), because the original is gated and I had no access token. Sampling: instruct tests with the base Qwen3.8-27B card's non-thinking values (temperature 0.7, top_p 0.8, top_k 20, presence penalty 1.5, because the Kaitchup card names none), GPQA with thinking with the same card's thinking values (temperature 1.0, top_p 0.95, top_k 20). These are my own tests, and no other run of this setup is mixed into the quality numbers on this page. The author tested this model on an RTX PRO 6000 Blackwell with 96 GB, and his serve command sets a context of 262,144 tokens. That does not fit this 32 GB card: after loading the 23.7 GB of weights, vLLM reports 73,728 tokens as the most it can hold. So speed, quality, the memory test and sparkDash ran at a context of 74,752 tokens, the largest size that loaded. His command also did not start on 32 GB at all: by default vLLM's start-up check sets aside a minimum cache for 512 sequences (12.25 GiB, no matter how long the context is), so I added one flag, --max-num-seqs 8. The cache is bf16 (his card names no cache type). Nothing else in his command was changed.

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts mmlupro --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts gpqa --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts ifeval --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts lcb --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts ruler --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts ruler --job nacht2410

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts think --job nacht2410

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. Measured 2026-10-25. Raw files: result of every step · log · contract.

Note from the run: The memory test ran on the first setting (--max-num-seqs 8, bf16 cache). Steps from 4,096 to 74,752 tokens: for each size the server was started again with that size as context, one prompt that fills about 95 % of it was sent, and the model answered at every step. The memory number is the peak of used GPU memory: 27.29 GiB from 4,096 to 65,536 tokens (the cache is set aside when the server starts), 29.54 GiB at 74,752 tokens, which is the largest size vLLM could hold with the weights loaded (the 32 GB card has about 31.8 GiB). The author's 262,144 tokens do not fit. The speed of decoding at the largest size was not measured in this test. The build test ran twice. In the first try, at the context of 74,752 tokens, the model's thinking used up the whole answer limit of one request (41,302 tokens, the room the context left), so it never wrote a page: 0 points, and that try is not shown on this page. For the second try I started the server again with more room for one long request: context 126,976 tokens, --max-num-seqs 1 and --gpu-memory-utilization 0.95 (cache still bf16). The second try is the one that counts and the one on this page. Speed, quality and sparkDash were not repeated; they stay at the first setting.

bash tools/quality2/measure_all.sh tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json --parts fit --job nacht2410

Also made this possible

• Base model (Qwen3.8-27B)QwenHugging Face @Alibaba_Qwen GitHub Website
• Engine (vLLM)vLLMGitHub

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5090 (32 GB), one card.

Model

kaitchup/Qwen3.8-27B-SLQ-EAR98 (SLQ EAR>98, Humming-packed, 23.73 GB)

Quantization (SLQ EAR>98, Humming) and serve command

huggingface.co/kaitchup/Qwen3.8-27B-SLQ-EAR98

Base model (Qwen3.8-27B)

huggingface.co/Qwen/Qwen3.8-27B

Engine

SLQ EAR>98 quant, Humming kernels, one RTX 5090 (32 GB), context 74,752 tokens (126,976 for the build test) — the program that loads the model and answers requests · github.com/vllm-project/vllmBuild and settings we used: vLLM 0.31.0 + humming-kernels 0.1.16 (pip wheels, CUDA 13.0 torch), bf16 KV, no speculation

Steps

1. Get the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory.

   Copy

   vllm serve kaitchup/Qwen3.8-27B-SLQ-EAR98 \
     --quantization humming \
     --dtype bfloat16 \
     --max-model-len 74752 \
     --limit-mm-per-prompt '{"image":0,"video":0}' \
     --enable-auto-tool-choice \
     --tool-call-parser qwen3_coder \
     --reasoning-parser qwen3

vLLM 0.31.0 + humming-kernels 0.1.16 (pip wheels, CUDA 13.0 torch), bf16 KV, no speculation; vllm serve kaitchup/Qwen3.8-27B-SLQ-EAR98 --quantization humming --dtype bfloat16 --max-model-len 74752 --limit-mm-per-prompt '{"image":0,"video":0}' --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

Mia's own benchmark, run unchanged

This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section on this page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.

Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-25. Version 2.50.0, commit 21a3511.

QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).

Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.

Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.

Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.

Note from the run: sparkDash runs its own tests (its own QA and MMLU questions at temperature 0, thinking off, a cut-off answer counts as wrong; no QA answer and 17 of the 285 MMLU answers were cut off at the answer limit and count as wrong), so its decode and prefill values are not comparable with Speed v2 above: its decode test uses a structured prompt, Speed v2 a code and a prose prompt. The sparkDash client ran on another machine and was never part of the measured group. It ran on the first setting (context 74,752).

python3 tools/quality2/sparkdash_client.py tools/quality2/cards/kaitchup-qwen38-27b-slq-ear98-vllm0310-5090.json <box> --client <client machine> --job nacht2410 --out $RUN

tool-eval-bench --base-url http://127.0.0.1:<port> --model kaitchup/Qwen3.8-27B-SLQ-EAR98 --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/nacht2410-sparkdash-r5/tooleval/result.json --no-live