KRAFTON's Raon-1 mini preview: IQ4_XS GGUF with the DFlash2 drafter

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds; long prompts (later supplementary run of 2026-11-11, same setup): prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests. Measured 2026-10-22 (long prompts: 2026-11-11). Raw files: speed result · speed contract · log · long answer · speed result · speed contract · log.

Requests at once: the server has 1 slot (from the author's recipe). C2, C4, C8 exceed the slots: the extra requests wait in the queue, and they were measured anyway. Other processes used up to 103 % CPU during the run. I ran the server command of the maker's model card (KRAFTON/Raon-1-mini-preview-gguf at revision a1f8b55, README lines 456 to 488) with the maker's settings: window 262,144 tokens, one slot, temperature 1.0, top_p 0.95, top_k 20, presence_penalty 1.5, reasoning on, DFlash2 drafting with 7 draft tokens. The engine is the maker's own llama.cpp branch, raon-1-mini-preview, at commit fb895662. The model card says to build from that branch because upstream support is pending (line 419), and the maker's repository has no release package (its list of releases is empty, checked on 10 Nov 2026), so I built it myself from the source, with CUDA 13.3, in an NVIDIA PyTorch container.

Differences from the maker's example, all named: the example loads the Q4_K_M file (18.33 GiB); I used the IQ4_XS file of the same repository (14.97 GiB, the smallest of its quants), because the Q4_K_M file alone is larger than the 16 GB of this card. The model files, the server flags and the 262,144-token window are the maker's; to make the model run on 16 GB I added -ngl 999 with --n-cpu-moe 9, which keeps the expert weights of the first 9 layers in system RAM (9 is the smallest value that loaded; I did not lower the window or any other setting of the maker). I also added --host 127.0.0.1 and --port 8888 (the example uses port 8000), --metrics (my tool reads the draft counters from it), --no-webui, -fit off, -t 8 and --cache-ram 0 (no prompt cache in system RAM, to leave the RAM to the machine's other services), and I named the drafter file with --spec-draft-model, where the example lets the server download it (dflash-raon-1-mini-preview-Q8_0.gguf, 699 MiB). The machine has a second GPU; the server was limited to the RTX 5060 Ti (CUDA_VISIBLE_DEVICES=1).

The headline 42.1 tok/s is the mean of the code prompt (39.9) and the prose prompt (44.3), each the median of two runs (spread 1.6 % and 11.1 %; both below my limit of 15 %). A first speed run of the same setup was repeated, because its spread was above my limit of 15 % for both prompts; this page shows the repeat, and the first run is not published here. Thinking was off in every speed request (the card says the model is designed for thinking mode and that switching it off is not recommended, README lines 151 and 152), and the tool's cache check found no cached prompt tokens in any of the 45 requests it examined. The drafter's guesses were accepted at a rate of 17.7 % (1,126 of 6,370 drafted tokens), with a mean accepted length of 2.2 tokens. I measured with sampling (the card's settings, seed 42) and 512 answer tokens per request. The card's own speed table (up to 2.18 times the speed of plain decoding with DFlash2) is for a single NVIDIA B300 with vLLM and the MXFP8 model, so it is another setup and not comparable with this page; I have not repeated the maker's test.

Requests at once: the server has 1 slot (--parallel 1, the maker's setting), so 2, 4 and 8 requests at once do not run in parallel; the extra requests wait in the queue. I measured them anyway, as the tool does. The total stays between 33.7 and 35.8 tok/s for 1, 2, 4 and 8 requests, and the speed of one request falls from 35.8 to 7.8 tok/s. The total at 1 request is lower than the decode headline because that test counts the whole answer time, including the wait for the first token, and uses the prose prompt only.

Long prompts: the long-prompt table is a run of its own, made after the speed run (this run: 11 Nov 2026, 14:44 to 15:27; the speed run above is from 22 Oct 2026). The server was started anew with the same start command, the same engine commit and the same sampling file (I compared the records of both runs: they are identical), so the window is the maker's 262,144 tokens with 1 slot. It has four steps of 16K, 32K, 64K and 128K tokens (a filler log with the code task or the prose task at its end, counted exactly: 16,384, 32,768, 65,536 and 131,072 tokens), two runs of the code prompt and two of the prose prompt per step, 512 answer tokens each, thinking off, no prompt cache (the tool's cache check found no cached prompt token in any of the 17 requests it examined). All 16 requests ended at the limit of 512 tokens and counted, and no step was reduced or left out, so every cell is filled, up to the full prompt of 131,072 tokens at 128K. The tool flagged the prose decode values of the 16K, 32K and 64K steps as noisy, because the spread (CV) of their two runs is above my limit of 15 % (16.8 %, 25.7 % and 16.3 %); the code decode values have a spread of at most 5.4 %, the prose decode value at 128K a spread of 3.2 %, and the prefill and first-token values a spread of at most 0.2 %. The prefill values at 2K, 8K and 32K of the first table (585.7, 585.6 and 532.0 tok/s) are from the speed run of 22 Oct 2026 and stay as they are: the 32K step of the long-prompt table is a separate measurement of its own. The memory test below stopped at 16,384 tokens because of the 90 % rule of my tool, not because the server failed. The long-prompt steps went above that: the server answered prompts of up to 131,072 tokens in the window of 262,144 tokens. The memory test below was not run for these sizes, and the long-prompt run has no memory reading of the measured card.

The machine: it was my workstation and ran its own background services during the test, and the speed test of this run had no load guard (the guard that repeats a step when another process is busy left no record in its contract); it only recorded the busiest other process every 10 seconds (83 samples; the speed contract holds the number of samples and the highest value). I counted the rest from the run's load record, which is not published here. In 23 of the 83 samples one other process was above 25 % of one core, up to 103 %: 14 times my file-indexing process (once a re-indexing script), 7 times a video-download helper, and once each a Docker status call and an ffmpeg process. At the start of the run the machine's other GPU, which the test did not use, was in use by another service of mine. So the speed numbers were not measured on an otherwise idle machine.

The long-prompt run (11 Nov) ran with the load guard on: a sample every 0.5 s, a step is thrown away and repeated when one process stays above 25 % of a core for 2 samples in a row or a foreign job shows up, and the box's own background services are on the guard's ignore list. It repeated none of its 16 steps in 5,188 samples. The busiest other process of a single step was a git backup job of the box's own files in 7 steps (up to 138.1 % of a core, in the first code run of the 128K step), the kernel's memory-compaction thread (kcompactd0) in 8 steps (8.0 to 22.0 %) and a browser process once (50.1 %). The guard's own sample record (not published here) has 23 readings of another process above 25 % of a core, in 21 of those samples (9 of a find process, 9 of a git backup job, 3 of a browser process, 1 of a worker of a search service of the box and 1 of kcompactd0); the same process was never above 25 % in two samples in a row, which is the condition for a repeat. The tool's separate quiet check (one sample every 10 seconds, no ignore list) saw 13 of 259 samples above 25 % of one core, up to 172.0 %: in the 16K step a git backup job (172.0 %) and a defunct Docker process (100.0 %); in the 32K step an index rebuild of the box's own services (96.7 %) and the box's own indexing service (101.0 %); in the 64K step the indexing service twice (106.0 and 100.0 %); in the 128K step the indexing service 4 times (104.0, 90.1, 58.6 and 101.0 %), an index rebuild (95.9 %) and a find process twice (100.0 and 100.0 %). Nothing was repeated or removed. The GPU reading of that quiet check is not that of the measured card: it stays at 5,602 MiB, the memory use of the machine's other GPU, which another service of mine was using at the start and at the end of the run.

python3 speed2.py --base-url http://127.0.0.1:39771/v1 --label speed-r5 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $BENCH/runs/raon-iq4xs/raw/ --contract $BENCH/runs/raon-iq4xs/raw/ --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source KRAFTON/Raon-1-mini-preview-gguf README.md (local copy $HOME/models/raon-1-mini-preview-gguf-a1f8b55/README.md, the storage box, revision a1f8b55) :154-171 (settings shared by all tasks: temperature 1.0, top_p 0.95, top_k 20, repetition_penalty 1.0, frequency_penalty 0, min_p 0, truncate_history_thinking false; repetition 1.0 and frequency 0 are the server defaults and are set by the server flags --repeat-penalty 1.0 --frequency-penalty 0.0, not sent per request (the run_row card lock only knows temperature/top_p/top_k/min_p/presence_penalty); presence_penalty 1.5 for 'everything else (chat, summarization, knowledge Q&A, agents, etc.)', 0 for coding / math), server command :464-470. The maker documents one profile only (thinking mode, :151-152 'designed for thinking mode; disabling it is not recommended'): the instruct profile (enable_thinking false, switch in the maker's own examples :266, :398) uses the same values. KNOWN DEVIATION: the maker says presence_penalty 0 for coding / math (:168-170, :478); our sampling file has one profile per family, so LiveCodeBench (coding) runs with 1.5 as well. --model Raon-1-mini-preview --no-cache-prompt --server-log $HOME/engines-raon/logs/raon.server.log --engine-version-file $BENCH/runs/raon-iq4xs/logs/engine-version.txt --server-slots 1 --server-slots-source KRAFTON/Raon-1-mini-preview-gguf README.md (local copy $HOME/models/raon-1-mini-preview-gguf-a1f8b55/README.md, the storage box, revision a1f8b55) :462 (--parallel 1) in the llama-server command :456-488

# Long prompts: supplementary run of 2026-11-11 (raw folder raon-ctx-1111-speed-r0-ctx)
python3 speed2.py --base-url http://127.0.0.1:39699/v1 --label raon-ctx-1111-speed-r0 --prompts $BENCH/repos/<speed-test-tool>/prompts-v2.json --output $RUN/raon-ctx-1111-speed-r0/raon-ctx-1111-speed-r0/speed2 --contract $RUN/raon-ctx-1111-speed-r0/raon-ctx-1111-speed-r0/speed2/speed-contract.json --parts ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --sampling {"temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source KRAFTON/Raon-1-mini-preview-gguf README.md (local copy $HOME/models/raon-1-mini-preview-gguf-a1f8b55/README.md, the storage box, revision a1f8b55) :154-171 (settings shared by all tasks: temperature 1.0, top_p 0.95, top_k 20, repetition_penalty 1.0, frequency_penalty 0, min_p 0, truncate_history_thinking false; repetition 1.0 and frequency 0 are the server defaults and are set by the server flags --repeat-penalty 1.0 --frequency-penalty 0.0, not sent per request (the run_row card lock only knows temperature/top_p/top_k/min_p/presence_penalty); presence_penalty 1.5 for 'everything else (chat, summarization, knowledge Q&A, agents, etc.)', 0 for coding / math), server command :464-470. The maker documents one profile only (thinking mode, :151-152 'designed for thinking mode; disabling it is not recommended'): the instruct profile (enable_thinking false, switch in the maker's own examples :266, :398) uses the same values. KNOWN DEVIATION: the maker says presence_penalty 0 for coding / math (:168-170, :478); our sampling file has one profile per family, so LiveCodeBench (coding) runs with 1.5 as well. --model Raon-1-mini-preview --no-cache-prompt --server-log $HOME/engines-raon/logs/raon.server.log --engine-version-file $RUN/engine-version.txt --server-slots 1 --server-slots-source KRAFTON/Raon-1-mini-preview-gguf README.md (local copy $HOME/models/raon-1-mini-preview-gguf-a1f8b55/README.md, the storage box, revision a1f8b55) :462 (--parallel 1) in the llama-server command :456-488 --server-ctx 262144 --server-ctx-source card box-5-raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti field ctx --supplement-of run

How well it answers

All seven quality tests ran. Six ran with thinking off and one (GPQA 30) with thinking on; the maker says the model is designed for thinking mode. Most answers of MMLU-Pro and GPQA Diamond were cut off at the 48-token limit and count as wrong. The notes below say how the numbers were taken.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-22.

Note from the run: MMLU-Pro and GPQA Diamond ran with an answer limit of 48 tokens (letter only, my usual setting; same questions, seed 42, the first letter of the answer counts); IFEval allowed 4,096 tokens per answer, long text 128, LiveCodeBench 32,768 and GPQA 30 with thinking 16,384. GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 questions), because the original is gated and no token was available. Three answer files are not published, because the site's filter held them back: the MMLU-Pro answers (an answer on optics contains the words resolving power, and the filter reads the word power as a power reading), the IFEval answers (a heading in one answer contains a word that the filter reads as the name of a machine) and the LiveCodeBench answers (a program has a variable named power). The summaries, the logs and the contracts of these tests are published, and the numbers in the table were counted from the complete files.

Cut-off answers count as wrong, and this matters most for two tests: in MMLU-Pro 185 of 400 answers and in GPQA Diamond 109 of 198 answers were cut off at the 48-token limit (the model often writes an explanation instead of only the letter), 2 of 200 in IFEval, 0 in LiveCodeBench, 7 in long text (2 at 8K, 5 at 32K) and 12 of the 30 GPQA answers with thinking (limit 16,384 tokens). I did not score the cut-off answers by their first letter, so the MMLU-Pro and GPQA Diamond results are the results of this rule with a 48-token limit. The total of 7 that the measuring tool wrote for run inst-r6 counts only LiveCodeBench and long text; this table counts every test. The median answer of the thinking test used 10,072 tokens and 220.4 seconds.

Thinking: the model card says Raon-1 mini is designed for thinking mode and that disabling it is not recommended (lines 151 and 152). The six tests with thinking off run outside the mode the model was designed for, and I have not measured how much that changes the numbers. Only GPQA 30 ran with thinking on: 16 of 30 right, and 12 of the 30 answers were cut off at 16,384 tokens, which count as wrong.

Sampling: the card gives presence_penalty 1.5 for everything except coding and math, where it says 0 (lines 168 to 171, and the note at line 478). I ran every test with 1.5, including LiveCodeBench, because my sampling file has one profile per model; so LiveCodeBench ran with a presence penalty that differs from the maker's coding setting. A separate rerun of LiveCodeBench with presence_penalty 0 was made on 22 Oct 2026; it is not part of this page, because its raw files are not published here.

Raon-1 mini preview is described by its makers as strong in Korean conversation and agentic tasks (model card, line 49). My tests are in English and none of them is an agent or a Korean test, so they do not measure that strength.

LiveCodeBench and long text (8K, 32K) are from a second quality run (inst-r6). The first run (inst-r2) started the same two tests after IFEval, but the server stopped answering during LiveCodeBench: that test scored 7.5 % in that run (3 of 40, in the contract of inst-r2), and long text failed with HTTP error 502 (also in that contract). I started the server again and ran both tests again in inst-r6, and the table shows that run; the contract and the log of inst-r2 hold the failed first attempt. MMLU-Pro, GPQA Diamond and IFEval are from the first run, before the server stopped; the thinking test is from a third run (think-r7).

Foreign load: the quality tests have no load guard; their quiet check records the busiest other process every 10 seconds (that load record is not published here; the contract of each run names the highest value). The machine ran my other background services during all of them. Samples above 25 % of one core: in the first run 93 of 347, up to 1,236 % (one sample of a Python script of my own tools; most of the rest my file-indexing process and a git process); in the second run 60 of 506, up to 107 % (my file-indexing process, a video-download helper); in the thinking test 36 of 603, up to 200 % (one sample of a video-watch script; mostly my file-indexing process).

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts mmlupro --job plan-2110

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts gpqa --job plan-2110

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts ifeval --job plan-2110

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts lcb --job plan-2110

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts ruler --job plan-2110

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts ruler --job plan-2110

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts think --job plan-2110

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the GPU memory in use (nvidia-smi); the GPU was empty before each start. First size that failed: 32K (32,768 tokens), up after 0 s. Measured 2026-11-10. Raw files: result of every step · log · contract.

Note from the run: This memory test was measured on 10 Nov 2026 with the same card (start script and settings) as the tests of 22 Oct 2026. The server was started fresh for each size, with the maker's flags and only the window changed (--ctx-size = the size; --n-cpu-moe 9 stays the same), and one prompt that fills about 95 % of the window was sent (96 answer tokens at most). The largest size that loaded and answered is 16,384 tokens (prompt 14,959 tokens, peak 12.91 GiB, decode 34.8 tok/s). At 32,768 tokens the server also loaded (peak 13.05 GiB) and answered normally (96 tokens, decode 34.3 tok/s), but the prompt had 29,486 tokens, which is 89.98 % of the window; my tool counts a step as complete only when the prompt fills at least 90 % of the window (29,491.2 tokens at 32,768), so this step is listed as not answered and 16,384 tokens is the largest size it counted. That is a limit of my tool, not a failure of the server; the real limit is probably higher, and I did not test it: the tool stops at the first size it does not count, so 65,536 tokens and above were not started. The memory number is the GPU memory in use on the RTX 5060 Ti (nvidia-smi); the expert weights of the first 9 layers sit in system RAM, which this test does not measure. The server's own window in the speed run is the maker's 262,144 tokens; this test does not show that this window loads with a full prompt. The decode speed in the table is the speed of that one answer (96 tokens).

bash tools/quality2/measure_all.sh tools/quality2/cards/raon1-mini-preview-iq4xs-dflash2-llamacpp-fb895662-5060ti.json --parts fit --job gpu1010-raon-fit

How it plays the Gauntlet

One solo race, season seed 21, one attempt, no help.

Game rank as of Nov 10, 2026. The rank is the order of the Gauntlet finish times in this memory class. Race page and raw files.

The race ran the recipe named on its race page, with the sampling and answer limit of its lane config. The engine build of the race is not recorded in the race files.

Result (result.json) Race log (log.jsonl) Model answers (io.jsonl) System prompt Exact command Verify output Model config Race page

Also made this possible

• Model (Raon-1 mini preview)KRAFTON AI@Krafton_AI Hugging Face GitHub Website
• Model files (IQ4_XS GGUF and DFlash2 drafter Q8_0, revision a1f8b55)KRAFTON AI@Krafton_AI Hugging Face GitHub Website
• Engine (KRAFTON's llama.cpp fork, branch raon-1-mini-preview, commit fb895662, built from source)KRAFTON AI@Krafton_AI GitHub Hugging Face Website
• Engine base (llama.cpp)ggml-orgGitHub @ggml_org Hugging Face Website
• Speculative decoding method (DFlash)z-labGitHub @zhijianliu_ Hugging Face

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

1x NVIDIA GeForce RTX 5060 Ti 16 GB, with the expert weights of the first 9 layers in system RAM (--n-cpu-moe 9)

Model

KRAFTON Raon-1 mini preview (28.8B MoE, 4.3B active) IQ4_XS GGUF + DFlash2 drafter Q8_0 (KRAFTON/Raon-1-mini-preview-gguf @a1f8b55)

Model (Raon-1 mini preview)

huggingface.co/KRAFTON/Raon-1-mini-preview

Model files (IQ4_XS GGUF and DFlash2 drafter Q8_0, revision a1f8b55)

huggingface.co/KRAFTON/Raon-1-mini-preview-gguf/tree/a1f8b55fbe76fc2fab264ecd51c21623ebc6ca0b

Engine

llama.cpp, KRAFTON's fork (branch raon-1-mini-preview, commit fb895662), one RTX 5060 Ti 16 GB — the program that loads the model and answers requests · github.com/krafton-ai/llama.cpp/commit/fb895662d25868756a19253ca5e0688f99b18994Build and settings we used: llama.cpp (KRAFTON fork, branch raon-1-mini-preview) @fb895662 (source build, CUDA 13.3, sm_120a), DFlash2 speculative decoding

Recipe (the server command of the model card, lines 456 to 488)

huggingface.co/KRAFTON/Raon-1-mini-preview-gguf/blob/a1f8b55fbe76fc2fab264ecd51c21623ebc6ca0b/README.md by KRAFTON AI

Context window

262,144 tokens: how much text the model can see at once

Steps

1. Get the recipe, the engine and the model files from the list above.

2. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   CUDA_VISIBLE_DEVICES=1 (docker \
     --gpus device=1) llama-server \
     -m raon-1-mini-preview-IQ4_XS.gguf \
     --alias Raon-1-mini-preview \
     --host 127.0.0.1 \
     --port 8888 \
     --ctx-size 262144 \
     --parallel 1 \
     --n-predict 32768 \
     --temp 1.0 \
     --top-p 0.95 \
     --top-k 20 \
     --min-p 0.0 \
     --repeat-penalty 1.0 \
     --frequency-penalty 0.0 \
     --presence-penalty 1.5 \
     --reasoning on \
     --chat-template-kwargs {"enable_thinking":true,"truncate_history_thinking":false} \
     --spec-type draft-dflash \
     --spec-draft-model dflash-raon-1-mini-preview-Q8_0.gguf \
     --spec-draft-n-max 7 \
     --spec-draft-sampling probabilistic \
     -ngl 999 \
     --n-cpu-moe 9 \
     -fit off \
     --cache-ram 0 \
     -t 8 \
     --no-webui \
     --metrics

   What the parts mean

   • CUDA_VISIBLE_DEVICES=1which NVIDIA GPU the program may use: the ID of the GPU, its number or the UUID that nvidia-smi -L prints
   • -m raon-1-mini-preview-IQ4_XS.ggufthe model file it loads
   • --ctx-size 262144the context window in tokens: how much text the model can see at once
   • -ngl 999how many layers of the model go on the GPU (999 means all of them)
   • --n-cpu-moe 9keeps the expert weights of this many layers in system RAM, so a model bigger than the GPU memory still runs
   • --parallel 1how many requests it answers at the same time
   • --temp 1.0sampling temperature: how random the answers are
   • --top-p 0.95top-p sampling: only the most likely next tokens that together reach this probability are considered
   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --cache-ram 0the most system RAM, in MiB, used to keep earlier prompts for reuse (0 turns this off)
   • -fit offturns off llama.cpp's automatic adjustment of unset settings to the free GPU memory; the command's settings are used as given
   • -t 8how many CPU threads it uses
   • --no-webuiturns off the built-in chat web page; only the API answers
   • --metricsturns on a /metrics page with server statistics (Prometheus format)
   • --chat-template-kwargs {"enable_thinking":true,"truncate_history_thinking":false}extra settings for the model's chat template, as JSON (enable_thinking false: answers without thinking)
   • --spec-…settings for speculative decoding: a draft guesses the next tokens and the model checks them (--spec-type: the kind of draft, --spec-draft-n-max: the most tokens drafted per step)
   • --alias Raon-1-mini-previewthe model name clients use in their requests

llama.cpp (KRAFTON fork, branch raon-1-mini-preview) @fb895662 (source build, CUDA 13.3, sm_120a), DFlash2 speculative decoding; CUDA_VISIBLE_DEVICES=1 (docker --gpus device=1) llama-server -m raon-1-mini-preview-IQ4_XS.gguf --alias Raon-1-mini-preview --host 127.0.0.1 --port 8888 --ctx-size 262144 --parallel 1 --n-predict 32768 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.0 --frequency-penalty 0.0 --presence-penalty 1.5 --reasoning on --chat-template-kwargs {"enable_thinking":true,"truncate_history_thinking":false} --spec-type draft-dflash --spec-draft-model dflash-raon-1-mini-preview-Q8_0.gguf --spec-draft-n-max 7 --spec-draft-sampling probabilistic -ngl 999 --n-cpu-moe 9 -fit off --cache-ram 0 -t 8 --no-webui --metrics