Hy4-Flash 220B on one RTX 5090: Project Maya v1.1.68 with the Maya-L quant (128K window)
Weights: Maya-L GGUF quant, 89.5 GB (made by Peasant Smith, peasantsmith/Hy4-Flash-Maya-GGUF, of tencent/Hy4-flash)
1x NVIDIA GeForce RTX 5090 32 GB (one GPU). Host: AMD Ryzen 9 9950X (16 cores, 32 threads), 92 GB RAM, NVMe. Driver 610.43.02, kernel 6.12.95+deb13-amd64
Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.
How fast it runs
| Measurement | Result |
|---|---|
| Decode | 20.1 tok/s |
| Words per second | 14.6 words/s |
| Characters per second | 99.0 chars/s |
| Decode, code prompt | 20.5 tok/s |
| Decode, prose prompt | 19.7 tok/s |
| Spread of the decode runs (CV) | 5.5 % |
| Spread of the code runs (CV) | 2.3 % |
| Spread of the prose runs (CV) | 5.5 % |
| Decode, window 1 | 20.3 tok/s |
| Prefill at 2K | 363.4 tok/s |
| Prefill at 8K | 483.9 tok/s |
| Prefill at 32K | 543.0 tok/s |
| Time to first token at 2K | 6.1 s |
| Time to first token at 8K | 18.4 s |
| Time to first token at 32K | 61.8 s |
| Draft acceptance rate | 61.1 % |
| 1 request at once (total) | 16.5 tok/s |
| 2 at once (total) | 17.0 tok/s |
| 4 at once (total) | 17.0 tok/s |
| 8 at once (total) | 16.7 tok/s |
| 1 request at once (per request) | 16.5 tok/s |
| 2 at once (per request) | 12.8 tok/s |
| 4 at once (per request) | 7.1 tok/s |
| 8 at once (per request) | 3.7 tok/s |
| Server slots (the engine answers one request at a time; the setup offers no setting) | 1 slot |
Long prompts
| Prompt length | Prefill | Time to first token | Decode, code prompt | Decode, prose prompt |
|---|---|---|---|---|
| 16K tokens | 528.3 tok/s | 31.0 s | 20.9 tok/s | 21.1 tok/s |
| 32K tokens | 524.9 tok/s | 62.4 s | 20.2 tok/s | 19.6 tok/s |
| 64K tokens | 538.8 tok/s | 121.6 s | 21.6 tok/s | 20.1 tok/s |
| 128K tokens | 534.6 tok/s | 243.2 s | 19.7 tok/s | 19.3 tok/s |
Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds; long prompts (later supplementary run of 2026-11-20, same setup): prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests. Measured 2026-11-20. Raw files: speed result · speed contract · log · speed result · speed contract · log.
What ran. Project Maya v1.1.68 (mw00/project-maya, tag v1.1.68, commit 05e0e2da90cacf174d791b0192f3488a3c61100b, the release as the author ships it: cloned and installed with the project's own setup script), Tencent's Hy4-Flash (220B parameters in total, 12B active) in the Maya quant Maya-L (89.5 GB), with the drafting block (MTP) switched on as the setup chooses it. The experts of the model are split between the card and the CPU: the engine's start line reads 'CPU lane: 2829 of 20992 routed experts in VRAM (13%), the rest on 16 CPU threads from the model's file'. The only difference to the page of the 32K window is the window itself: the setup offers the stage 128K (./setup.sh --context 131072; its own recommendation for this card is 32K), so that prompts of 32K, 64K and 128K fit. Everything else is the setup's default. The longer window takes memory from the expert pool: the card holds 2,829 of 20,992 routed experts (13 %) here against 3,239 (15 %) at 32K, and the rest of the experts is computed on the CPU. The quant is 89.5 GB (89,512,229,920 bytes) plus a 63.4 GB n-gram table and a 7.5 GB drafting block. The headline 20.1 tok/s is the mean of the code prompt (20.5) and the prose prompt (19.7), medians of two runs each (spread (CV) 2.3 % and 5.5 %). The code prompt has 1,218 and 1,219 tokens and the prose prompt 1,299 and 1,302 tokens; every reply is 512 tokens long and ends at the limit. About 34.3 s for 500 words (14.58 words per second). Accepted drafts: 61.1 % of the drafted tokens (1,171 of 1,918). Long prompts. The context steps (decode on the code prompt after a prompt of that size) at 16K, 32K, 64K and 128K tokens, as the table shows them (from the supplement run on the same server settings): prefill 528.3, 524.9, 538.8 and 534.6 tok/s, first token after 31.0 s, 62.4 s, 121.6 s and 243.2 s, decode 20.9, 20.2, 21.6 and 19.7 tok/s. The 128K step is measured at 130,048 tokens (the window minus 1,024); its decode on the prose prompt is 19.3 tok/s. The main run measured the steps at 16K, 32K and 64K too, with prefill 565.8, 531.5 and 540.4 tok/s and first token after 29.0 s, 61.7 s and 121.3 s; both runs are in the register and the raw files. The prefill series of the main run has the sizes 2K, 8K and 32K (363.4, 483.9 and 543.0 tok/s). Two runs, one setup. The main run ended at the 128K step: another process on the host (the container service, cause not established) used more than 25 % of a core in repeated peaks and the load guard threw the step away until its limit, so the run ended there. The supplement ran the context steps again on the same server settings and finished clean. The page shows the supplement's context steps (16K to 128K) and the main run's other values. Requests at once. Hy4-Flash answers one request at a time in Project Maya (docs/SETTINGS.md line 2385), so the levels 2, 4 and 8 at once are above its one slot: the requests wait in a queue and run one after another, not in parallel. The total stays near the value of one request (16.5, 17.0, 17.0 and 16.7 tok/s with 1, 2, 4 and 8 requests) and the speed of each request falls as the queue grows (16.5, 12.8, 7.1 and 3.7 tok/s). Sampling. The speed test sent the sampling of the model card, which is the same for instruct and thinking (temperature 0.9, top_p 1.0), with the flag for no thinking (enable_thinking false), seed 42 and cache_prompt false; Tencent's card names reasoning_effort high (thinking) or no_think, and the test uses no_think. Cache check: the check of the test tool is marked not ok. 41 of the 59 requests that it checked report cached tokens: 6 or 7 tokens on the decode and requests-at-once requests and 22 or 23 tokens on the prefill requests. The Maya server keeps the first tokens of the chat template (the template head) in its cache on every request, so a fully cold prompt is not possible with this server; each request still carries a nonce, so no earlier answer or long prompt body is reused. The result is shown as measured. Foreign load. The load guard of the speed test repeated 7 of 37 steps (10 repeats) (details in the guard section of the speed result). Main run: 7 steps repeated (10 repeats) of 37; supplement: 2 steps repeated (2 repeats) of 16.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 speed2.py --base-url http://127.0.0.1:44395/v1 --label hy4-maya-l-128k-2011-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/hy4-maya-l-128k-2011-speed-r0/hy4-maya-l-128k-2011-speed-r0/speed2 --contract $RUN/hy4-maya-l-128k-2011-speed-r0/hy4-maya-l-128k-2011-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,ctx --prefill-depths 2048,8192,16384,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 0.9, "top_p": 1.0} --sampling-source tencent/Hy4-flash README (recommended temperature=0.9, top_p=1.0) and generation_config.json; Maya docs/SETTINGS.md:2343 --model hy4-flash --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source mw00/project-maya @05e0e2da: Hy4-Flash answers one request at a time (docs/SETTINGS.md:2385, CHANGELOG.md v1.1.68 Known limits). --server-ctx 131072 --server-ctx-source card box-6-hy4-flash-maya-l-mayahy4-1x-v1168-1911-128k field ctx
# Long prompts: supplementary run of 2026-11-20 (raw folder hy4-maya-l-128k-2011-ctx-speed-r0-ctx)
python3 speed2.py --base-url http://127.0.0.1:32789/v1 --label hy4-maya-l-128k-2011-ctx-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/hy4-maya-l-128k-2011-ctx-speed-r0/hy4-maya-l-128k-2011-ctx-speed-r0/speed2 --contract $RUN/hy4-maya-l-128k-2011-ctx-speed-r0/hy4-maya-l-128k-2011-ctx-speed-r0/speed2/speed-contract.json --parts ctx --prefill-depths 2048,8192,16384,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 0.9, "top_p": 1.0} --sampling-source tencent/Hy4-flash README (recommended temperature=0.9, top_p=1.0) and generation_config.json; Maya docs/SETTINGS.md:2343 --model hy4-flash --no-cache-prompt --server-log $RUN/server-A.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source mw00/project-maya @05e0e2da: Hy4-Flash answers one request at a time (docs/SETTINGS.md:2385, CHANGELOG.md v1.1.68 Known limits). --server-ctx 131072 --server-ctx-source card box-6-hy4-flash-maya-l-mayahy4-1x-v1168-1911-128k field ctx --supplement-of hy4-maya-l-128k-2011Quality
Not run on this setup: no Quality v2 test, so the page shows no quality number.
Also made this possible
- NVIDIA engine for Hy4 (maya-hy4, contributed to Project Maya in pull requests #1040 and #1051)NneedmorevramGitHub
- Model (Hy4-Flash, 220B parameters in total, 12B active)TTencentHugging Face
- Quants (Maya-L and Maya-XS GGUF files)PpeasantsmithHugging Face
Run it yourself
What you need and how to start it, step by step. The results below come from exactly this setup.
What you need
- Hardware
- 1x NVIDIA GeForce RTX 5090 32 GB (one GPU). Host: AMD Ryzen 9 9950X (16 cores, 32 threads), 92 GB RAM, NVMe. Driver 610.43.02, kernel 6.12.95+deb13-amd64
- Model
- peasantsmith/Hy4-Flash-Maya-GGUF Maya-L (Project Maya's own quant of tencent/Hy4-flash, 220B)
- Model (Hy4-Flash, 220B parameters in total, 12B active)
- huggingface.co/tencent/Hy4-flash
- Quants (Maya-L and Maya-XS GGUF files)
- huggingface.co/peasantsmith/Hy4-Flash-Maya-GGUF
- Engine
- Project Maya as shipped, with the setup's 128K stage; half of the model runs on the CPU; Speed v2 only — the program that loads the model and answers requests · github.com/mw00/project-mayaBuild and settings we used: Project Maya maya-hy4 v1.1.68 (mw00/project-maya @05e0e2da), CUDA 13.1, sm_120
Steps
Get the engine and the model files from the list above.
Start the server with this command. It loads the model into memory.
bash <start script of the run>
Start text for this setupfor reproducing↓
You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.
Project Maya maya-hy4 v1.1.68 (mw00/project-maya @05e0e2da), CUDA 13.1, sm_120; bash <start script of the run> (starts: .venv/bin/python maya.py --config maya-hy4-flash-maya-l.json --port <port> --plain; config written by ./maya.sh --setup --context 131072 for the model hy4-flash-maya-l; engine args as written by the setup: --pack <Maya-L folder> --max-context 131072 --mtp)Context and sampling: 131,072 tokens (the setup's stage 128K, ./setup.sh --context 131072; its recommendation for this card is 32K), prompt chunk chosen by the engine (auto), drafting on (MTP), vision on, one request at a time. Speed test with thinking off per request, sampling of the model card (temperature 0.9, top_p 1.0), seed 42. The 128K step is measured at 130,048 tokens.. Hardware: 1x NVIDIA GeForce RTX 5090 32 GB (one GPU). Host: AMD Ryzen 9 9950X (16 cores, 32 threads), 92 GB RAM, NVMe. Driver 610.43.02, kernel 6.12.95+deb13-amd64. All raw files of this setup: raw/v2/hy4-flash-5090-maya-l-project-maya-1168-ctx131072-mw00.