Home / 32 GB

32 GB · 2× RTX 5060 Ti 16 GB (32 GB in total)

Hy4-Flash 220B on two RTX 5060 Ti: Project Maya v1.1.68 with the Maya-XS quant (chunk 1,024, drafting off)

Weights: Maya-XS GGUF quant, 44.8 GB (made by Peasant Smith, peasantsmith/Hy4-Flash-Maya-GGUF, of tencent/Hy4-flash)

2x NVIDIA GeForce RTX 5060 Ti 16 GB (two GPUs, layer split). Host: Intel Core i7-12700K (20 threads), 31 GiB RAM, NVMe.

Engine and setup (Project Maya v1.1.68, commit 05e0e2da) and the Maya quants of Hy4-FlashPeasant Smith (mw00)X @PeasantSmith
Project Maya maya-hy4 v1.1.68 (mw00/project-maya @05e0e2da), local build with user CUDA 13.3 toolkit, sm_120, drafting off (no --mtp), --chunk 1024commit mayaproject 05e0e2da2 GPU (layer split)measured 2026-11-20

Not ranked yet. The rank needs a finished, verified Gauntlet race (the game test), and this setup has not played it yet. It shows Speed v2, Quality v2 and the memory test, every number with its raw files and exact command. How the game test works.

Speed v2

How fast it runs

Decode
15.8tok/s
Decode, code
16.0tok/s
Decode, prose
15.7tok/s
500 words
≈ 44.2s
Prefill at 8K
187.4tok/s
First token at 2K
14.2s
MeasurementResult
Decode15.8 tok/s
Words per second11.3 words/s
Characters per second77.0 chars/s
Decode, code prompt16.0 tok/s
Decode, prose prompt15.7 tok/s
Spread of the decode runs (CV)13.0 %
Spread of the code runs (CV)13.0 %
Spread of the prose runs (CV)1.7 %
Decode, window 115.9 tok/s
Prefill at 2K155.0 tok/s
Prefill at 8K187.4 tok/s
Time to first token at 2K14.2 s
Time to first token at 8K44.6 s
1 request at once (total)12.8 tok/s
2 at once (total)14.8 tok/s
4 at once (total)15.5 tok/s
8 at once (total)15.7 tok/s
1 request at once (per request)12.8 tok/s
2 at once (per request)10.8 tok/s
4 at once (per request)6.4 tok/s
8 at once (per request)3.5 tok/s
Server slots (the engine answers one request at a time; the setup offers no setting)1 slot

Long prompts

Prompt lengthPrefillTime to first tokenDecode, code promptDecode, prose prompt
16K tokens167.1 tok/s98.1 s13.6 tok/s14.4 tok/s

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-20. Raw files: speed result · speed contract · log · long answer.

What ran. Project Maya v1.1.68 (mw00/project-maya, tag v1.1.68, commit 05e0e2da90cacf174d791b0192f3488a3c61100b, the release as the author ships it: cloned and installed with the project's own setup script), Tencent's Hy4-Flash (220B parameters in total, 12B active) in the Maya quant Maya-XS (44.8 GB), with the drafting block (MTP) switched OFF (the config has no --mtp, so there are no drafts and no acceptance rate) as the setup chooses it. The experts of the model are split between the card and the CPU: the engine's start line reads 'CPU lane: 10455 of 20992 routed experts in VRAM (50%), the rest on 12 CPU threads from the model's file'. Maya-XS (44.8 GB, 44,786,173,984 bytes), the smaller Maya quant of Hy4-Flash; Tencent's model card lists the Maya-XS at 88.5 % and the Maya-L at 96.6 % 'same top token as the original', so the two quants are not the same file and the numbers of the pages of the RTX 5090 are not the numbers of this quant. Two settings differ from the setup's defaults, both documented by the author: the prompt chunk is set by hand to 1,024 tokens (--chunk 1024, docs/SETTINGS.md line 2531; the setup's default chose 4,096) and drafting is off (the config leaves out --mtp; docs/SETUP.md line 214 and docs/SETTINGS.md lines 2374 to 2384 describe the switch). The smaller chunk leaves room for more experts on the cards: 10,455 of 20,992 routed experts (50 %) against 4,469 (21 %) with the setup's defaults. The engine runs the layer split over the two cards (30.0 GB of VRAM over 2 GPUs), the window is 32,768 tokens.

Note on the raw metadata of this run. The register and the raw metadata files of this run carry an older label: they say drafting on, one GPU and the setup's default arguments. The run itself used both cards, no drafting and --chunk 1024, as the engine's start line and the config of the run show (engine args --pack <folder> --max-context 32768 --chunk 1024, two GPUs); the page uses these values. The numbers are the measured ones, confirmed by two independent checkers against the raw files.

The headline 15.8 tok/s is the mean of the code prompt (16.0) and the prose prompt (15.7), medians of two runs each (spread (CV) 13.0 % and 1.7 %). The code prompt has 1,216 and 1,219 tokens and the prose prompt 1,301 and 1,303 tokens; every reply is 512 tokens long and ends at the limit. About 44.2 s for 500 words (11.31 words per second).

Compared with the page of the setup's defaults (same cards, same quant, same box): decode 15.8 tok/s here against 6.7 tok/s there, prefill at 8K 187.4 tok/s against 124.5 tok/s. The chunk-1,024 setup is faster in decode, where half of the experts sit on the cards, and its prefill is higher too, although a chunk of 1,024 is smaller than the default 4,096.

Long prompts. The window is 32,768 tokens: the prefill series has the sizes 2K and 8K (155.0 and 187.4 tok/s, first token after 14.2 s and 44.6 s) and the 16K context step ran (prefill 167.1 tok/s, first token after 98.1 s); the steps at 32K, 64K and 128K do not fit the window and were not sent.

Requests at once. Hy4-Flash answers one request at a time in Project Maya (docs/SETTINGS.md line 2385), so the levels 2, 4 and 8 at once are above its one slot: the requests wait in a queue and run one after another, not in parallel. The total stays near the value of one request (12.8, 14.8, 15.5 and 15.7 tok/s with 1, 2, 4 and 8 requests) and the speed of each request falls as the queue grows (12.8, 10.8, 6.4 and 3.5 tok/s).

Sampling. The speed test sent the sampling of the model card, which is the same for instruct and thinking (temperature 0.9, top_p 1.0), with the flag for no thinking (enable_thinking false), seed 42 and cache_prompt false; Tencent's card names reasoning_effort high (thinking) or no_think, and the test uses no_think. Cache check: the check of the test tool is marked not ok. 41 of the 45 requests that it checked report cached tokens: 6 or 7 tokens on the decode and requests-at-once requests and 22 or 23 tokens on the prefill requests. The Maya server keeps the first tokens of the chat template (the template head) in its cache on every request, so a fully cold prompt is not possible with this server; each request still carries a nonce, so no earlier answer or long prompt body is reused. The result is shown as measured.

Foreign load. The load guard of the speed test repeated 0 of 26 steps (0 repeats).

Exact command we measured withfor reproducing↓

You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.

Speed v2 (run hy4-maya-xs-c1024-2011d-speed-r0)
python3 speed2.py --base-url http://127.0.0.1:32949/v1 --label hy4-maya-xs-c1024-2011d-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/hy4-maya-xs-c1024-2011d-speed-r0/hy4-maya-xs-c1024-2011d-speed-r0/speed2 --contract $RUN/raw/hy4-maya-xs-c1024-2011d-speed-r0/hy4-maya-xs-c1024-2011d-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 0.9, "top_p": 1.0} --sampling-source Tencent Hy4-flash generation_config.json as applied by Project Maya (maya_quants.MODEL_DEFAULTS['hy4']; docs/SETTINGS.md:2343-2352: temperature 0.9, top-p 1.0, no top-k) and the model card peasantsmith/Hy4-Flash-Maya-GGUF 'Recommended Settings'. Greedy is discouraged by the author (hf-modelcard). --model hy4-flash --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 1 --server-slots-source docs/SETTINGS.md:2385 (Hy4-Flash: --parallel not available, one request at a time) and serve/server.py:7 (one sequence at a time behind a FIFO) --server-ctx 32768 --server-ctx-source card box-5-hy4-flash-maya-xs-mayahy4-v1168-2x5060ti-2011-chunk1024-nodraft field ctx
Quality v2

Quality

Not run on this setup: no Quality v2 test, so the page shows no quality number.

Credits

Also made this possible

  • NVIDIA engine for Hy4 (maya-hy4, contributed to Project Maya in pull requests #1040 and #1051)needmorevramGitHub
  • Model (Hy4-Flash, 220B parameters in total, 12B active)TencentHugging Face
  • Quants (Maya-L and Maya-XS GGUF files)peasantsmithHugging Face
Setup

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware
2x NVIDIA GeForce RTX 5060 Ti 16 GB (two GPUs, layer split). Host: Intel Core i7-12700K (20 threads), 31 GiB RAM, NVMe.
Model
peasantsmith/Hy4-Flash-Maya-GGUF Maya-XS (Project Maya's own quant of tencent/Hy4-flash, 220B, sha256 equal to the pinned table of v1.1.68)
Model (Hy4-Flash, 220B parameters in total, 12B active)
huggingface.co/tencent/Hy4-flash
Quants (Maya-L and Maya-XS GGUF files)
huggingface.co/peasantsmith/Hy4-Flash-Maya-GGUF
Engine
Project Maya with two documented settings changed (prompt chunk 1,024, drafting off); half of the experts on the cards; Speed v2 only — the program that loads the model and answers requests · github.com/mw00/project-maya

Steps

  1. Get the engine and the model files from the list above.

  2. Start the server with this command. It loads the model into memory.

    bash <start script of the run>
Start text for this setupfor reproducing↓

You do not need this to run the model. It is the start text of this setup, based on the setup card of this run, with local paths, machine names and addresses shortened or replaced by placeholders, notes added on some pages, and the engine build in front, so anyone can repeat the result.

Start, based on the setup card of this run
Project Maya maya-hy4 v1.1.68 (mw00/project-maya @05e0e2da), local build with user CUDA 13.3 toolkit, sm_120, drafting off (no --mtp), --chunk 1024; bash <start script of the run> (starts: .venv/bin/python maya.py --config <config with the args below> --port <port> --plain, with CUDA_VISIBLE_DEVICES set to the two cards; engine args: --pack <Maya-XS folder> --max-context 32768 --chunk 1024; no --mtp)

Context and sampling: 32,768 tokens, prompt chunk 1,024 tokens (--chunk 1024), drafting off (no --mtp), vision on, one request at a time. Speed test with thinking off per request, sampling of the model card (temperature 0.9, top_p 1.0), seed 42. The raw metadata files of this run carry an older label (drafting on, one GPU); the page uses the values of the run itself.. Hardware: 2x NVIDIA GeForce RTX 5060 Ti 16 GB (two GPUs, layer split). Host: Intel Core i7-12700K (20 threads), 31 GiB RAM, NVMe.. All raw files of this setup: raw/v2/hy4-flash-2x5060ti-maya-xs-project-maya-1168-chunk1024-no-drafting-mw00.