Unsloth's Mistral Large 4 recipe: UD-IQ1_S with the DSpark draft

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-10-31. Raw files: speed result · speed contract · log · long answer.

Setup: Unsloth's UD-IQ1_S quant of Mistral Large 4 (6 files, 233 GB for 1,049 billion parameters, about 1.8 bits per weight, our calculation: file size times 8 divided by the parameters) with the DSpark draft file from the same Hugging Face repository, on llama.cpp b12020 with Metal, built from the upstream source, not Unsloth's build. Changed from Unsloth's guide: context 65,536 tokens instead of 32,768, the DSpark draft from the Hugging Face card (the guide's command has no draft), and the flags -ngl 99, --fit off and --cache-ram 0. Thinking cannot be turned off: the chat template only has the reasoning efforts low, medium and high, and we ran high. So every answer cut at 512 tokens ends inside the reasoning, and the decode speed counts those reasoning tokens. In the 16K and 32K rows of the long-prompt table, the first token is the first reasoning token. The words-per-second values and the 500-words line are not shown, because no answer has a visible word at 512 tokens. The 32K first-token time appears twice: 242.3 s in the long-prompt table (chat requests) and 237.2 s in the prefill test (median of 3 runs, plain completion requests). 16K code decode is noisy: the two runs gave 22.5 and 29.7 tok/s. 64K and 128K did not fit the server context of 65,536 tokens, so there are no rows for them. Requests at once: the server has 4 slots that share one KV pool. 8 at once is above the 4 slots, so the extra requests wait in line; it was measured anyway. The long answer is one long HTML page: 32,446 tokens at 21.8 tok/s with 2.59 tokens per draft step; 86 % of its characters were reasoning. The load monitor of the speed tool was on. It voids a request and repeats it when another process holds over 25 % CPU for 2 samples in a row, or when a download, build, ffmpeg or rsync job shows up. In this run 5 of 30 steps were repeated and none ended invalid. Short spikes stay in: every step saw a spike from another process, 30 to 92 % of one core. Exception: 11 macOS system services (Spotlight, iCloud, asset downloads and similar) are not counted as load, so they may have used CPU during steps. We added this exception after mobileassetd at 97 % CPU and an ffmpeg job voided an earlier run of this test. The Photos and Spotlight analysis daemons were paused during the run.

python3 speed2.py --base-url http://127.0.0.1:65039/v1 --label ml4-dspark-mac-3110-s-speed-r1 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/ml4-dspark-mac-3110-s-speed-r1/ml4-dspark-mac-3110-s-speed-r1/speed2 --contract $RUN/ml4-dspark-mac-3110-s-speed-r1/ml4-dspark-mac-3110-s-speed-r1/speed2/speed-contract.json --parts decode,prefill,conc,l1,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 0, "min_p": 0.0} --sampling-source Mistral card mistralai/Mistral-Large-4-1T-A52B and unsloth guide https://unsloth.ai/docs/models/mistral-large-4 (temperature 1.0, top_p 0.95 for all reasoning efforts; guide adds --top-k 0 --min-p 0.0); reasoning_effort high is the server default of the guide command (--chat-template-kwargs), speed2 sends no own value. --model Mistral-Large-4-1T-A52B-UD-IQ1_S --no-cache-prompt --server-log $RUN/logs/server.log --engine-version-file $RUN/engine-version.txt --server-slots 4 --server-slots-source ggml-org/llama.cpp@0ec4027a tools/server/server.cpp:126-129 (n_parallel auto = 4 slots, kv_unified true); common/arg.cpp:1353 (n_parallel default -1 = auto); the unsloth command sets no -np, ours sets none either. The 4 slots share one KV pool of -c tokens (unified KV). --server-ctx 65536 --server-ctx-source card mac-mistral-large-4-1t-a52b-ud-iq1s-dspark-llamacpp-0ec4027-m5ultra-3110 field ctx

How well it answers when it thinks

The two fixed quality lists that allow 48 tokens per answer (MMLU-Pro and GPQA Diamond, one letter each) cannot measure this model. It always thinks, so its answers stop inside the reasoning before a letter appears. Those lists are not shown, and this setup has no rank. The test below gives the model room to think: 30 GPQA questions with a limit of 16,384 tokens per answer.

Thinking on, reasoning effort high. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-31.

Note from the run: 11 of the 30 answers reached the limit of 16,384 tokens before they gave a letter. They count as wrong, so 60.0 % is the score under this limit. 4 questions were sent at a time (the server has 4 slots), so the time per question is the time under that load. The median tokens and seconds are taken over all 30 questions, cut-off ones included.

bash tools/quality2/measure_all.sh tools/quality2/cards/mistral-large-4-1t-a52b-ud-iq1s-dspark-llamacpp-0ec4027-m5ultra-3110.json --parts think --job ml4-dspark-mac-3110-t

Also made this possible

• ModelMistral AIHugging Face GitHub Website
• Engine (llama.cpp b12020; Mistral Large 4 support by ngxson, pull request 31376)ggml-org (llama.cpp)@ggml_org GitHub Hugging Face Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

Mac Studio M5 Ultra, 256 GB unified memory, one box

System

macOS (the Mac above)

Model

mistralai/Mistral-Large-4-1T-A52B (1T total, 52B active) unsloth GGUF UD-IQ1_S @1eb4373d + DSpark draft Q8_0 (same repo)

Quant and DSpark draft files (GGUF) and the run guide

huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF

Engine

llama.cpp b12020 (upstream build, Metal) on one Mac Studio M5 Ultra, Unsloth UD-IQ1_S with the DSpark draft, context 65,536 tokens — the program that loads the model and answers requests · github.com/ggml-org/llama.cpp/releases/tag/b12020

Steps

1. Get the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   llama-server \
     -m Mistral-Large-4-1T-A52B-UD-IQ1_S-00001-of-00006.gguf \
     -md dspark-Mistral-Large-4-1T-A52B-Q8_0.gguf \
     --spec-type draft-dspark \
     --spec-draft-n-max 3 \
     --alias Mistral-Large-4-1T-A52B-UD-IQ1_S \
     -ngl 99 \
     --fit off \
     -c 65536 \
     --cache-ram 0 \
     --temp 1.0 \
     --top-p 0.95 \
     --top-k 0 \
     --min-p 0.0 \
     --chat-template-kwargs '{"reasoning_effort":"high"}' \
     --host 127.0.0.1 \
     --port 8888 \
     --no-webui \
     --metrics

   What the parts mean

   • -m Mistral-Large-4-1T-A52B-UD-IQ1_S-00001-of-00006.ggufthe model file it loads
   • -c 65536the context window in tokens: how much text the model can see at once
   • -ngl 99how many layers of the model go on the GPU (999 means all of them)
   • --temp 1.0sampling temperature: how random the answers are
   • --top-p 0.95top-p sampling: only the most likely next tokens that together reach this probability are considered
   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --cache-ram 0the most system RAM, in MiB, used to keep earlier prompts for reuse (0 turns this off)
   • --no-webuiturns off the built-in chat web page; only the API answers
   • --metricsturns on a /metrics page with server statistics (Prometheus format)
   • --chat-template-kwargs '{"reasoning_effort":"high"}'extra settings for the model's chat template, as JSON (reasoning_effort: how much the model thinks before it answers: low, medium or high)
   • --spec-…settings for speculative decoding: a draft guesses the next tokens and the model checks them (--spec-type: the kind of draft, --spec-draft-n-max: the most tokens drafted per step)
   • --alias Mistral-Large-4-1T-A52B-UD-IQ1_Sthe model name clients use in their requests

llama.cpp b12020 @0ec4027a (upstream, built from source Oct 31, 2026, Metal, arm64), single box, F16 KV, DSpark draft dspark-Mistral-Large-4-1T-A52B-Q8_0.gguf (draft-dspark, n_max 3); -c 65536; llama-server -m Mistral-Large-4-1T-A52B-UD-IQ1_S-00001-of-00006.gguf -md dspark-Mistral-Large-4-1T-A52B-Q8_0.gguf --spec-type draft-dspark --spec-draft-n-max 3 --alias Mistral-Large-4-1T-A52B-UD-IQ1_S -ngl 99 --fit off -c 65536 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 0 --min-p 0.0 --chat-template-kwargs '{"reasoning_effort":"high"}' --host 127.0.0.1 --port 8888 --no-webui --metrics