How I test

The machines

• 16 GB: RTX 5060 Ti 16 GB.
• 32 GB: RTX 5090 32 GB.
• 128 / 256 / 512 GB: one, two or four DGX Spark (NVIDIA GB10) boxes, linked by direct cables.
• 256 GB Mac: one Mac Studio M5 Ultra with 256 GB, as its own memory class.

Commands on the setup pages are shown normalized: internal bind addresses appear as 127.0.0.1, machine names appear as neutral labels, and remote wrappers (ssh) are stripped. The raw files are the originals with these changes: internal machine addresses (LAN IPs, machine names, home paths, user names) are replaced by neutral placeholders, power readings are removed, and German key names and notes are translated to English. The answers in the quality-test files are not changed.

The tests

• Speed v2. One warm-up, then two fixed prompts (code and prose), two runs each, 512 tokens per run. The number is the median; decode is also shown separately for the code prompt and the prose prompt, and the main number is the median of those two. Seed 42, thinking off, no prompt cache. Prefill and time to first token are measured at 2K, 8K and 32K. “≈ s for 500 words” is the time for a 500-word answer at the measured decode speed. CV is the spread between the runs; a measurement above 15 % is repeated.
• Quality v2. Fixed question lists, seed 42, thinking off: MMLU-Pro 400, GPQA Diamond 198 (answer is one letter), IFEval 200, LiveCodeBench 40 (the code is run), long text at 8K and 32K (4 tasks × 10). GPQA 30 runs once more with thinking on (limit 16,384 tokens). The sampling comes from the model card. ± is the 95 % margin of error (Wilson interval) in percentage points. An answer cut off at the token limit counts as wrong; the number of cut answers stands next to the result.
• Memory test. For each context size (4K, 8K, 16K and so on, up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer. Peak memory is the memory in use at that size.
• Build test. The model builds one 3D scene as an agent, with one attempt and no help: an airship dock (16 and 32 GB) or a voxel world (128 GB and up). A browser checks the page, 0 to 10 points. Target 60 minutes, “finish now” after 90, hard stop after 120. The harness and its version are shown with every result. A failed run is explained on its page.
• sparkDash. Mia's own benchmark tool, run unchanged: quality QA and MMLU, Tool Eval Quick, decode with one request and prefill. It has its own section and its own column, and it is never mixed with my numbers.
• Specialist models. A model that its model card calls a specialist (for example trained for cybersecurity) is marked. My tests do not measure that strength.

When a run is shown

• Every part above is done, every number has its raw file and its exact command, and an independent check passed.
• The setup runs the way its author or community runs it best (harness and settings are on its page).
• No other job ran on the machine during the run.
• A run is ranked only after I have rated the page it built. Without my rating it is “not ranked yet” and there is no pick.

The overall score

Every run that is on the site and that I have rated gets one overall score from 0 to 100. The run with the highest score in a memory class is rank 1 of that class. In a class with two or more ranked runs, rank 1 is also my pick. A run that is alone in its class is rank 1, but there is no pick until a second run is ranked.

• Formula. Overall = 100 × (0.45 × S + 0.3 × Q + 0.05 × B + 0.2 × N), rounded to one decimal.
• S, speed. Decode speed (tokens per second, one request, Speed v2) divided by the best decode speed of the class.
• Q, quality. The mean of seven percentages (MMLU-Pro 400, GPQA 198, IFEval 200, LiveCodeBench 40, long text 8K, long text 32K and GPQA 30 with thinking) divided by the best mean of the class.
• B, browser check. The points of the build test divided by 10.
• N, my rating. My own rating of the page the setup built, divided by 10.
• The numbers are the ones shown on the setup page. sparkDash, the memory test and speculative-decoding numbers are not part of the score. Different runs of one setup are never mixed.

Assumption. The weights (speed 45 %, quality 30 %, browser check 5 %, my rating 20 %) are my rule. The way I turn them into one number is my assumption: each part is measured against the best run of the same class, the quality part is the plain mean of the seven percentages, and equal scores are ordered by decode speed. Scores are relative to the best run in the class today; a better model later moves older runs down. The score is shown with the date of the newest measurement in the class (for example “as of 20 Oct 2026”).

Numbers are rounded to one decimal; the raw files hold the full values. A dash means: not measured yet.