Meta's Muse Glimmer 30B on one RTX 5060 Ti 16 GB: UD-IQ3_XXS with Meta's DFlash drafter

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds; long prompts (later supplementary run of 2026-11-11, same setup): prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests. Measured 2026-11-11 (long prompts: supplementary run, 2026-11-11). Raw files: speed result · speed contract · log · long answer · speed result · speed contract · log.

Requests at once: the server has 1 slot (-np 1: the box limit of the 16 GB card, see slots_source; the maker's example uses 4 slots). C2, C4, C8 exceed the slots: the extra requests wait in the queue, and they were measured anyway. Other processes used up to 124.1 % CPU during the run. What ran. Meta's Muse Glimmer 30B (29.6B parameters, dense, 52 layers, sliding-window attention in a 3:1 pattern) as the UD-IQ3_XXS quant of Unsloth (unsloth/Muse-Glimmer-30B-GGUF, revision faa5b025, 13.1 GB) with Meta's DFlash drafter (meta-models/Muse-Glimmer-30B-GGUF, Q4_K_M, revision b44f1072), served by the official llama.cpp release package b11972 of ggml-org (commit cf24317bea, CUDA 13.4 build for Linux x86_64, dated 28 Oct 2026; the archive checksums equal the digests that GitHub lists). The engine was not built by us. The weights and the drafter were checked against the Hugging Face checksums on 29 Oct 2026. Where this run differs from the maker's example (README, section 'Run': -ngl 99 -c 131072 -np 4 --jinja --temp 1.0 --top-p 0.95 --top-k 64). (1) Quant: the maker's own K-Quant-17GB file (16.8 GB) does not fit a 16 GB card together with the drafter, so the largest Unsloth UD file that fits with the drafter and a window of at least 32K was used (fit test). (2) Window and slots: 98,304 tokens and one slot (-np 1) instead of 131,072 tokens and four slots; the 131,072-token window did not load on this card in the box test of 29 Oct 2026 (cudaMalloc out of memory in the compute buffers; that test ran in a container built from the same commit). So the 128K step of the context test does not fit and has no value, the levels of 2, 4 and 8 requests at once are above the one slot and queue (they were measured anyway), and an answer that needs more than 98,304 tokens counts as wrong. (3) The prompt cache in RAM is off (--cache-ram 0) to protect the memory of other services on the machine. (4) Drafts: the maker's line sets no draft length; we asked for 16 (the drafter's block size, --spec-draft-n-max 16) and the engine reduces it to 15, as its log says. (5) Text only: no --mmproj, no image input. (6) Port 8888 and the web interface off. Against the maker's own figures. Meta's README (lines 257 to 265) gives, for the K-Quant-17GB model with the drafter on an RTX 5090: 74.9 tok/s without speculation and 233.4 tok/s with DFlash (average over a diverse prompt set, batch size 1, greedy decoding, llama.cpp). These numbers were not measured by us. They are for a different card and a different file, and the README names no figure for a 16 GB card. We did not repeat the test in the maker's method (greedy, one stream) on this card; the speed test here uses sampling (temperature 1.0), 512 output tokens and reasoning text, so the two sets of numbers cannot be compared directly. The context steps exist twice: the main speed run (11 Nov 2026, 15:34 to 15:56) and a supplement run with the same setup (16:02 to 16:17) both measured 16K, 32K and 64K; the long-prompt table on this page shows the supplement run, and the raw files of both runs are linked. Prefill and first-token times agree within 0.1 %; the decode values differ by the spread of two runs per point (for example 27.4 and 50.5 tok/s for the 32K code step in the supplement), and the supplement flags four decode points with a spread above 15 % as noisy. Other services ran on the machine during the speed tests: the guard of the speed test saw foreign processes (git, an indexer) up to 124.1 % of one core in the main run and up to 108.0 % in the supplement, never above 25 % for two samples in a row in the main run; in the supplement it threw away and repeated 3 steps. The load and clock monitor of the two speed runs watched the other card of the machine, so there is no load or clock series of the measuring card for them; the prefill values of the two runs agree within 0.1 %, and no throttling is visible.

python3 speed2.py --base-url http://127.0.0.1:44215/v1 --label muse-rel-16gb-1111-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/muse-rel-16gb-1111-speed-r0/muse-rel-16gb-1111-speed-r0/speed2 --contract $RUN/muse-rel-16gb-1111-speed-r0/muse-rel-16gb-1111-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64} --sampling-source meta-models/Muse-Glimmer-30B-GGUF README.md (fetched Oct 29, 2026 02:2x, revision b44f1072cf75e918a1b82a3a17b45c4389b076e7; copy $HOME/bench/meta-gguf-README.md), Best Practices and llama-server example: temperature 1.0, top_p 0.95, top_k 64 (also generation_config.json of meta-models/Muse-Glimmer-30B @cec94fcf: temperature 1.0, top_p 0.95, top_k 64). min_p, presence and repeat penalties are not named by the maker: server defaults. The maker's template cannot switch reasoning off (README 'Reasoning cannot be switched off'); default reasoning_strength high. --model muse-glimmer-30B --no-cache-prompt --server-log $HOME/muse-rel-1111/logs/server.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source box limit: UD-IQ3_XXS (unsloth Dynamic) + drafter leave room for one slot only on 16 GB (fit test Oct 29, 2026, $HOME/bench/); the maker's example uses 4 slots --server-ctx 98304 --server-ctx-source card box-5-muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti field ctx

# Long prompts: supplementary run of 2026-11-11 (raw folder muse-rel-16gb-1111-ctx-speed-r0-ctx)
python3 speed2.py --base-url http://127.0.0.1:37375/v1 --label muse-rel-16gb-1111-ctx-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/muse-rel-16gb-1111-ctx-speed-r0/muse-rel-16gb-1111-ctx-speed-r0/speed2 --contract $RUN/muse-rel-16gb-1111-ctx-speed-r0/muse-rel-16gb-1111-ctx-speed-r0/speed2/speed-contract.json --parts ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 64} --sampling-source meta-models/Muse-Glimmer-30B-GGUF README.md (fetched Oct 29, 2026 02:2x, revision b44f1072cf75e918a1b82a3a17b45c4389b076e7; copy $HOME/bench/meta-gguf-README.md), Best Practices and llama-server example: temperature 1.0, top_p 0.95, top_k 64 (also generation_config.json of meta-models/Muse-Glimmer-30B @cec94fcf: temperature 1.0, top_p 0.95, top_k 64). min_p, presence and repeat penalties are not named by the maker: server defaults. The maker's template cannot switch reasoning off (README 'Reasoning cannot be switched off'); default reasoning_strength high. --model muse-glimmer-30B --no-cache-prompt --server-log $HOME/muse-rel-1111/logs/server.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source box limit: UD-IQ3_XXS (unsloth Dynamic) + drafter leave room for one slot only on 16 GB (fit test Oct 29, 2026, $HOME/bench/); the maker's example uses 4 slots --server-ctx 98304 --server-ctx-source card box-5-muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti field ctx --supplement-of muse-rel-16gb-1111

How well it answers

Thinking cannot be switched off for this model; the tests sent enable_thinking false and every answer still came back with reasoning text. A cut-off answer counts as wrong. Answer limits: 8,192 tokens for MMLU-Pro and GPQA (letter answer; the other pages use 48, which would cut every answer of this model), 4,096 for IFEval, 32,768 for LiveCodeBench, 128 for the long-text tasks, 16,384 for GPQA with thinking. Seed 42, sampling of the maker's README, 1 request at a time. Long text (8K and 32K): the limit of 128 tokens was used up by the reasoning before the answer in 39 of 40 questions at each length, so these two values (2.5 % each) measure the token limit and not the reading of the text; the repeat with a larger limit has not run. The instruct tests ran in pieces of at most 230 minutes. Pieces ended at the planned stop, not by a model error: MMLU-Pro was complete (400 of 400) before the first stop; the parts that the stops cut off were asked again in full in the later pieces. GPQA Diamond: the last 7 of the 198 questions had got only connection errors at the end of one piece (the server was stopped); they were asked again afterwards, all 7 answered. 191 answers come from the first card of the machine and 7 from a replaced 16 GB card, with the same weights, sampling and seed; the PCIe link of the first card was read when idle, not under load, and the replaced card's link was read under load. IFEval: 1 of the 200 requests got an HTTP 500 from the server (it could not read the model output in the expected format) and counts as wrong. The GPQA questions come from the public mirror fingertap/GPQA-Diamond (the original is gated and no token was available). Raw numbers only, no overall score.

The model's reasoning cannot be switched off (see the text above the table). Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-11 to 2026-11-12.

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts mmlupro --job muse-rel-16gb-1111

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts gpqa --job muse-rel-16gb-1111

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts ifeval --job muse-rel-16gb-1111

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts lcb --job muse-rel-16gb-1111

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts ruler --job muse-rel-16gb-1111

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts ruler --job muse-rel-16gb-1111

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts think --job muse-rel-16gb-1111

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. Measured 2026-11-11. Raw files: result of every step · log · contract.

bash tools/quality2/measure_all.sh tools/quality2/cards/muse-glimmer-30b-udiq3xxs-dflash-llamacpp-b11972-release-5060ti.json --parts fit --job muse-rel-16gb-1111

Also made this possible

• Base model (Muse-Glimmer-30B)MMetaHugging Face
• Quant (UD-IQ3_XXS GGUF, revision faa5b025)Unsloth AI@unslothai Hugging Face GitHub Website
• Engine (llama.cpp release b11972, commit cf24317bea, official CUDA 13.4 package)ggml-org@ggml_org GitHub Hugging Face Website
• DFlash method, on which the drafter is basedz-lab (Zhijian Liu)@zhijianliu_ Paper GitHub Hugging Face

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5060 Ti 16 GB (one machine, second card not used), driver 595.71.05, kernel 6.12.86 (Debian 13)

Model

Meta Muse Glimmer 30B (29.6B dense, 52 layers, sliding window 3:1) UD-IQ3_XXS GGUF (unsloth/Muse-Glimmer-30B-GGUF @faa5b025) + DFlash drafter Q4_K_M (meta-models/Muse-Glimmer-30B-GGUF @b44f1072)

Model, DFlash drafter and recipe (README of meta-models/Muse-Glimmer-30B-GGUF, revision b44f1072)

huggingface.co/meta-models/Muse-Glimmer-30B-GGUF

Base model (Muse-Glimmer-30B)

huggingface.co/meta-models/Muse-Glimmer-30B

Quant (UD-IQ3_XXS GGUF, revision faa5b025)

huggingface.co/unsloth/Muse-Glimmer-30B-GGUF

Engine

official llama.cpp release b11972 (CUDA 13.4 package), 98,304-token window, 1 slot — the program that loads the model and answers requests · github.com/ggml-org/llama.cpp/releases/tag/b11972Build and settings we used: llama.cpp b11972 @cf24317bea (official release package, CUDA 13.4, native), DFlash drafter n_max 16 (clamped to 15 by the engine), F16 KV

Steps

1. Get the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   CUDA_VISIBLE_DEVICES=<the 16 GB card> llama-server \
     -m Muse-Glimmer-30B-UD-IQ3_XXS.gguf \
     -md dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
     -ngld 99 \
     --spec-draft-n-max 16 \
     -a muse-glimmer-30B \
     -ngl 99 \
     -c 98304 \
     -np 1 \
     --host 127.0.0.1 \
     --port 8888 \
     --jinja \
     --temp 1.0 \
     --top-p 0.95 \
     --top-k 64 \
     --no-webui \
     --metrics \
     --cache-ram 0

   What the parts mean

   • CUDA_VISIBLE_DEVICES=<thewhich NVIDIA GPU the program may use: the ID of the GPU, its number or the UUID that nvidia-smi -L prints
   • -m Muse-Glimmer-30B-UD-IQ3_XXS.ggufthe model file it loads
   • -c 98304the context window in tokens: how much text the model can see at once
   • -ngl 99how many layers of the model go on the GPU (999 means all of them)
   • -np 1how many requests it answers at the same time
   • --jinjauses the chat template that comes with the model
   • --temp 1.0sampling temperature: how random the answers are
   • --top-p 0.95top-p sampling: only the most likely next tokens that together reach this probability are considered
   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --cache-ram 0the most system RAM, in MiB, used to keep earlier prompts for reuse (0 turns this off)
   • --no-webuiturns off the built-in chat web page; only the API answers
   • --metricsturns on a /metrics page with server statistics (Prometheus format)
   • --spec-…settings for speculative decoding: a draft guesses the next tokens and the model checks them (--spec-type: the kind of draft, --spec-draft-n-max: the most tokens drafted per step)

llama.cpp b11972 @cf24317bea (official release package, CUDA 13.4, native), DFlash drafter n_max 16 (clamped to 15 by the engine), F16 KV; CUDA_VISIBLE_DEVICES=<the 16 GB card> llama-server -m Muse-Glimmer-30B-UD-IQ3_XXS.gguf -md dflash-Muse-Glimmer-30B-Q4_K_M.gguf -ngld 99 --spec-draft-n-max 16 -a muse-glimmer-30B -ngl 99 -c 98304 -np 1 --host 127.0.0.1 --port 8888 --jinja --temp 1.0 --top-p 0.95 --top-k 64 --no-webui --metrics --cache-ram 0 (the llama-server line of the maker's README with the changes named on this page; engine: the official release package llama-b11972-bin-ubuntu-cuda-13.4-x64, not built by us)