Qwen3.8-Flash-Next oQ4e-mtp (oQ4e files at revision 2322c888, oMLX 0.7.3)

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-11. Raw files: speed result · speed contract · log.

Compared with the earlier page for this model on this Mac (measured on 19 Oct 2026), this page uses oMLX 0.7.3 instead of oMLX 0.7.0 and the oQ4e files at revision 2322c888 instead of the earlier copy of the files (the model card says the files were re-quantized with oMLX 0.7.3 on 29 Oct 2026), and each page shows only its own measurement.

Engine: oMLX 0.7.3 from the official release of Jun Kim's repository (jundot/omlx, tag v0.7.3, published on 31 Oct 2026): the wheel omlx-0.7.3-cp313-cp313-macosx_15_0_universal2.whl, sha256 9b09aa8f, installed by me, not built by me. The sha256 of the installed file equals the digest GitHub lists for that release file. The newest release on that page when the tests ran was v0.7.4.dev1 (4 Nov 2026), tagged as a development version; it was not used, because the model card names oMLX 0.7.3 (README.md, lines 20 and 22). Weights: Jundot/Qwen3.8-Flash-Next-oQ4e-mtp on Hugging Face at revision 2322c888 (29 Oct 2026, commit message: 'Re-quantize with the improved oQe quantization in oMLX 0.7.3'), a mixed-precision MLX checkpoint of Qwen/Qwen3.8-Flash-Next made with oMLX's oQ quantisation (Hugging Face user Jundot; the Hugging Face profile and the GitHub profile of the oMLX repository's owner both carry the name Jun Kim). The model card says: "Re-quantized with the improved oQe quantization in oMLX 0.7.3. Please re-download if you have an earlier copy." (README.md, line 20). I downloaded the 34 files of the repository (100.30 GiB) on 9 Nov 2026 (21:38 to 22:32); the size of every file equals the Hugging Face listing, and the sha256 of the 22 files that have one in the listing matched at the end of the download. No file has a later date than that download. The base model is Qwen/Qwen3.8-Flash-Next.

The model card names no start command and no settings, so there is no author recipe to follow here. The setup is oMLX's own start (the command is shown below, in the start text), with MTP (multi-token prediction) switched on for this model: mtp_enabled=true and mtp_fixed_depth=3 in model_settings.json, the same setting as on the earlier page; oMLX's default is mtp_enabled=false (omlx/model_settings.py:493; the depth field is line 502). The other settings are those of the settings file that oMLX created in the base folder of this install at its first start (10 Nov, 23:23). It holds the API key and is not published. The server accepts at most 8 requests at once (max_concurrent_requests, oMLX's default, omlx/settings.py:336); oMLX does not report a slot count, so the slots cell is empty and the tool measured every level of requests at once anyway. No acceptance rate is shown: the tool found no counters for drafted and accepted tokens on oMLX's metrics endpoint.

The headline 146.5 tok/s is the mean of the code prompt (154.0) and the prose prompt (139.1), medians of two runs each; the spread (CV) of the two code runs is 5.3 % and of the two prose runs 7.0 % (both under the 15 % limit of our tool). Thinking was off in every request, and the tool's cache check found no cached prompt tokens in any of the 60 requests it examined; the server log of the speed test (not published) has no prefix-cache restore line. Requests at once: 1, 2, 4 and 8 requests ran together. The total is 123.2 tok/s with 1 request, 93.0 with 2, 114.2 with 4 and 132.9 with 8, and one request gets 123.2, 47.0, 29.1 and 17.0 tok/s; the 1-request figure is measured over the whole request, so it is below the decode speed. Long prompts: the first token of the 128K-token prompt came after 26.7 s (prefill 4904.1 tok/s), see the table; the four first-token times of the 128K runs lie between 26.2 s and 27.2 s.

Load on the Mac in the speed test (11 Nov 2026, 06:59 to 07:09): the macOS desktop and other programs ran on the same Mac. The load guard (a sample every half second, 1,142 samples) threw away 4 tries and repeated 3 of its 37 steps, each because one foreign process used more than 25 % of one core for at least two samples in a row: the first round of the 2-requests-at-once step twice (Apple's Metal compiler service, up to 98 %, then an Apple scheduling service, duetexpertd, up to 106 %), the first round of the 8-requests-at-once step (Apple's Metal compiler service, up to 28 %) and the first code run of the 64K-token step (Apple's Metal compiler service, up to 92 %). Each repeat ran after 5 s of quiet, and the values on this page are from the repeated tries; in those three kept tries the busiest other process was launchd at 39.3 % (2 requests at once), launchd at 27.6 % (8 requests at once) and an Apple scheduling service, dasd, at 21.8 % (64K-token step, first code run). In the other 34 steps the guard saw no process above 25 % of one core for two samples in a row, and the busiest other process reached 64.0 % of one core at most. 17 of those steps had a process above 25 % in a sample; each is named here with its step and its highest value: Apple's Metal compiler service in 2 steps (4 requests at once, round 1, 64.0 %; prefill at 8K, run 2, 58.0 %); a Python process in 2 steps (prefill at 8K, run 3, 38.0 %; 128K-token prompt, code run 1, 32.0 %); an Apple system monitor service (sysmond) in 1 step (64K-token prompt, prose run 2, 26.0 %); launchd (the macOS service manager) in 12 steps (4 requests at once, round 2, 29.1 %; 32K-token prompt, code run 1, 28.3 %; 128K-token prompt, prose run 2, 28.2 %; 8 requests at once, round 2, 28.0 %; 16K-token prompt, code run 1, 28.0 %; 64K-token prompt, code run 2, 28.0 %; decode, code prompt, run 2, 28.0 %; 32K-token prompt, code run 2, 27.8 %; prefill at 32K, run 3, 27.7 %; 64K-token prompt, prose run 1, 26.1 %; 128K-token prompt, code run 2, 26.0 %; 128K-token prompt, prose run 1, 26.0 %). The guard does not count the programs on its ignore list, which the speed contract names: processes started with claude, bfs or ugrep, the macOS desktop shell, every program under /Applications or /System/Applications (a web browser, for example), the dictation helper, the oMLX server itself and a few other tools of mine. The quiet check (one sample every 10 seconds, 57 samples, no ignore list) saw no sample above 25 % of one core; the highest was 19.0 % (Apple's Metal compiler service), and the quiet logs before and after the run list no foreign problems. So the Mac was not idle, and I cannot show that nothing else used it during the test: the numbers are measured under this load.

Earlier tries produced no values and are not part of this page (they are not in the register, and their raw files are not published): a first speed try (10 Nov 2026, 23:23) ended after 17 seconds because oMLX answered 'model not found' (it listed two other models; the cause was not found, and every later start found the model); a second try (10 Nov, 23:25 to 23:35) was stopped by the load guard after 15 tries thrown away in 9 steps, the last four tries of the second code run of the 64K step each disturbed by an ffmpeg job (two ffmpeg jobs on voice sample files, a loudness check and the dictation helper); a third try (11 Nov, 02:09 to 02:13) was stopped by the guard in the third prefill run at 32K after 10 tries thrown away in 6 steps (the last four tries: Apple's file-event service, fseventsd, twice, up to 134 %; a cp process, 100 %; a Python process, 102 %). A first try of the quality tests (10 Nov, 23:24) failed on the same 'model not found' answer for every request (0.0 %), and it is not part of this page either. The speed run on this page is the first one that ran to its end under the guard.

Not published: the quiet logs and the sampling file of this run (the site's filter holds back files of that kind by name), the load guard's sample file and error file, oMLX's own server log, and the per-request capture folder (requests/: 150 files, 26 of them from tries that were thrown away). The speed contract, with the summaries of the load guard and of the quiet check and its record of each step, is published, so the foreign-load figures above can be checked only in part. In the published raw files the source notes (the line numbers of oMLX's code, local folders and machine names, the name of the speed-test tool's repository) were corrected or made neutral; no measured value was touched.

python3 speed2.py --base-url http://127.0.0.1:54281/v1 --label mac256-run-omlx5-s1-speed-r0 --prompts $BENCH/repos/speed-v2/prompts-v2.json --output $RUN/mac256-run-omlx5-s1-speed-r0/mac256-run-omlx5-s1-speed-r0/speed2 --contract $RUN/mac256-run-omlx5-s1-speed-r0/mac256-run-omlx5-s1-speed-r0/speed2/speed-contract.json --parts decode,prefill,conc,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/Qwen3.8-Flash-Next-oQ4e-mtp/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.7, "top_p": 0.8, "top_k": 20, "min_p": 0.0, "presence_penalty": 1.5} --sampling-source Qwen/Qwen3.8-Flash-Next README.md:343 (Instruct/non-thinking) and :342 (Thinking), local copy not published; generation_config.json (1.0/20/0.95) = thinking default only, not used for instruct --model Qwen3.8-Flash-Next-oQ4e-mtp --no-cache-prompt --server-log $LOGS/server-A.log --engine-version-file $LOGS/engine-version.txt --server-ctx 262144 --server-ctx-source card mac-flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110 field ctx

How well it answers

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured: MMLU-Pro 2026-11-10; GPQA Diamond 2026-11-10; IFEval 2026-11-10; LiveCodeBench 2026-11-11; Long text 8K 2026-11-11; Long text 32K 2026-11-11; GPQA 30, thinking on 2026-11-10.

Note from the run: MMLU-Pro (400 questions), GPQA Diamond (198), IFEval (200), LiveCodeBench (40 tasks) and RULER at 8K and 32K tokens (40 tasks each) ran with thinking off (enable_thinking false) and the Instruct profile of Qwen's model card (temperature 0.7, top-p 0.8, top-k 20, min-p 0.0, presence penalty 1.5), seed 42. One question at a time: the setup has no known slot count, so the tool's default of one request at a time applied. MMLU-Pro and GPQA ran with an answer limit of 1,024 tokens (the other pages use 48), letter only; IFEval with 4,096 tokens; LiveCodeBench with 32,768; RULER with 128. Every MMLU-Pro and GPQA reply had a letter that the tool could read, and the median reply is 1 token. 7 MMLU-Pro answers and 1 GPQA answer were cut off at the 1,024-token limit (counted wrong); no IFEval, LiveCodeBench or RULER answer was cut off. RULER by task at 8K and 32K: multi-key retrieval 10 of 10 and 10 of 10; common-word extraction 9 of 10 and 10 of 10; multi-hop question answering 8 of 10 and 4 of 10; variable tracking 5 of 10 and 0 of 10. Three of the 80 prompts were not exactly the target length (8,191 and 8,193 tokens at 8K, 32,767 tokens at 32K); the tool counts them offline with the model's own tokenizer.json. IFEval is scored with the official checker of Google Research (instruction_following_eval, commit 3ef52111), strict, at the level of the prompt. LiveCodeBench: the 40 newest tasks of the tool's list (tasks dated 2025-03-15 to 2025-04-06), one answer per task, the code runs in a container (python:3.12-slim); no task was lost to a sandbox error or to a missing code block. The thinking test (GPQA 30) ran with thinking on and the Thinking profile (temperature 1.0, top-p 0.95, top-k 20, min-p 0.0, presence penalty 0.0), a limit of 16,384 tokens (thinking and answer together): 7 of the 30 answers were cut off at the limit and count as wrong; the model thinks long: median 6,251 tokens and 39.0 s per question, one question at a time.

The first RULER attempt failed. It ran at the end of the first quality run (10 Nov 2026, about 23:48) and stopped with HTTP 422 (Unprocessable Content): the tool asked oMLX's /tokenize endpoint for a token count with a request field that oMLX 0.7.3 does not take (the endpoint takes a prompt or messages, omlx/api/openai_models.py, lines 597 and 606; the endpoint is at omlx/server.py:8406); the other parts of that run kept their results, and its published contract says that RULER failed. RULER was repeated as its own run (11 Nov 2026, 00:54 to 01:01), with the tool counting tokens offline with the checkpoint's tokenizer.json; the RULER numbers on this page are from that repeat. The tests ran in four runs (10 Nov, 23:35 to 23:48: MMLU-Pro, GPQA, IFEval; 23:48 to 00:13: the thinking test; 11 Nov, 00:14 to 00:41: LiveCodeBench; 00:54 to 01:01: RULER), each after a quiet check that found no foreign problems, with the oMLX server started anew for each run.

These quality tests have no foreign-load guard. The quiet check samples the busiest other process every 10 seconds, with no ignore list; the quiet logs before and after each run list no foreign problems. Samples above 25 % of one core by part (the parts are laid end to end from the run's durations in the contract, so a sample at a border may belong to the next part): MMLU-Pro: 8 of 18 samples, an Apple scheduling service (duetexpertd) once (up to 95.4 %); a web browser helper process 4 times (up to 53.3 %); the macOS desktop shell (WindowServer) 2 times (up to 37.6 %); a process started with the command claude once (up to 31.5 %); GPQA Diamond: none of 7 samples; IFEval: 3 of 48 samples, the Claude desktop app once (up to 94.9 %); a web browser helper process once (up to 42.2 %); the macOS desktop shell (WindowServer) once (up to 40.5 %); the thinking test (GPQA 30): 20 of 151 samples, the macOS desktop shell (WindowServer) 11 times (up to 114.6 %); an Apple scheduling service (duetexpertd) once (up to 93.7 %); the macOS System Settings agent once (up to 64.7 %); a web browser helper process 4 times (up to 62.0 %); the macOS audio service (coreaudiod) 2 times (up to 31.4 %); launchd (the macOS service manager) once (up to 27.6 %); LiveCodeBench: 34 of 156 samples, a file search (bfs), not part of the test 4 times (up to 593.9 %); a web browser helper process 15 times (up to 126.7 %); the macOS desktop shell (WindowServer) 4 times (up to 113.8 %); Apple's virtualization service (it runs the container machine of the LiveCodeBench sandbox) 4 times (up to 100.0 %); the macOS log archiver (bzip2 on a system log) 2 times (up to 99.3 %); a macOS metadata (Spotlight) worker once (up to 66.8 %); the network extension of a VPN application 2 times (up to 57.8 %); an Apple system monitor service (sysmond) once (up to 34.7 %); the macOS audio service (coreaudiod) once (up to 33.8 %); RULER at 8K: 3 of 10 samples, the macOS desktop shell (WindowServer) 2 times (up to 57.1 %); the macOS audio service (coreaudiod) once (up to 32.5 %); RULER at 32K: 8 of 27 samples, a file search (bfs), not part of the test 2 times (up to 594.6 %); a web browser helper process 3 times (up to 53.5 %); the macOS desktop shell (WindowServer) 2 times (up to 32.8 %); a process started with the command claude once (up to 26.1 %). The highest sample of each run, which the four contracts record, was 95.4 % in the instruction run, 114.6 % in the thinking run, 593.9 % in the LiveCodeBench run and 594.6 % in the RULER run. So the Mac was not idle during these tests: the numbers are measured under this load, and a 10-second sample does not show every short peak. GPQA Diamond: the original is gated and no Hugging Face token was available, so the public mirror fingertap/GPQA-Diamond (198 rows) was used.

Not published: the quiet logs and the sampling file of each run (the site's filter holds back files of that kind by name), the LiveCodeBench results file (lcb.jsonl; model code contains a word the filter reads as a hardware setting), the error output of the failed first RULER attempt (ruler.error.txt; it holds file paths of the test computer), the folders with the extracted programs and the sandbox runs of LiveCodeBench (lcb/code, 80 files; lcb/sandbox, 160 files) and the 80 prompt files of RULER (ruler/prompts). The summaries and logs of all tests, the contracts, and the answers of MMLU-Pro, GPQA, IFEval, RULER and the thinking test are published, and the numbers of the table were counted from the complete files. These are my own tests, and no other run of this setup is mixed into the numbers on this page.

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts mmlupro --job mac256-run-omlx2-q

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts gpqa --job mac256-run-omlx2-q

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts ifeval --job mac256-run-omlx2-q

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts lcb --job mac256-run-omlx2-lcb

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts ruler --job mac256-run-omlx2-qr

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts ruler --job mac256-run-omlx2-qr

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts think --job mac256-run-omlx2-q

How much context fits

Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the rise of used memory over the idle value before each server start (unified memory, so GPU and CPU share it). Measured 2026-11-11. Raw files: result of every step · log · contract.

Note from the run: The idle memory before the steps was not constant: 24.1 GiB before the first step, 24.2 GiB before the last one. Each peak is counted over the idle value before its own step. The memory test starts the server anew for each of the seven sizes (4K, 8K, 16K, 32K, 64K, 128K and 256K), with oMLX's own context setting, the per-model max_context_window (omlx/model_settings.py:363), set to that size, and sends one prompt of about 95 % of the size; the model had to answer. All seven sizes were answered (finish reason stop). The largest size tested is 262,144 tokens (a prompt of 249,729 tokens), the model's own window (config.json, max_position_embeddings 262144), so there is no larger size to test. The memory number is the rise of the Mac's used memory over its idle value before each server start (a Mac has one pool of memory for CPU and GPU), so it includes loading the model (the 34 weight files are 100.30 GiB); the Mac had between 22.8 and 24.8 GiB in use before the seven starts. The rise was 104.58 GiB at 4K, 108.39 at 8K, 107.26 at 16K, 112.38 at 128K and 118.80 at 256K, so it does not grow in every step. Because it is the rise of the whole Mac, other programs that used or released memory at the same time are inside the number. The decode speed at the largest size, 55.8 tok/s, comes from one answer of 77 tokens (the tool's stream timing), the prefill column is the speed the server reports, and the first-token column is the tool's own timing. The prefill speeds may not be cold values: oMLX's SSD prefix cache is on by default and was not emptied before this test. The server logs that are kept show it for two steps: at 64K the server restored 24,576 of the 62,733 prompt tokens from the cache and at 256K 122,880 of 249,729; the log of the 16K step has no restore line. The logs of the 4K, 8K, 32K and 128K steps no longer exist, because the start script names each server log after the date and time of the start (for example serve-1111-0041.log), and a later start with the same date and time in that name (it has no seconds) overwrote the earlier log, so for those four steps it is not known how much was restored. This test ran on 11 Nov 2026 (00:41 to 00:44), after the quality tests of the same night, with a quiet check before it that found no foreign problems and no foreign-load sampling during it. The start script's output for each step (server-c<size>.log; its text is German) and oMLX's own server logs are not published; fit.json, the test's output and its contract are published.

bash tools/quality2/measure_all.sh tools/quality2/cards/flashnext-oq4e-mtp-omlx-0.7.3-newweights-m5ultra-1110.json --parts fit --job mac256-run-omlx2-fit

Also made this possible

• oQ4e weights (Hugging Face user Jundot, revision 2322c888)Jun Kim@jundotkim Hugging Face GitHub Website
• Model (Qwen3.8-Flash-Next, built by the Qwen team)QwenHugging Face @Alibaba_Qwen GitHub Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

Mac Studio M5 Ultra 256 GB

System

macOS (the Mac above)

Model

Qwen3.8-Flash-Next oQ4e-mtp (Jundot, oMLX 0.7.3 oQe requant @2322c888)

oQ4e weights (Hugging Face user Jundot, revision 2322c888)

huggingface.co/Jundot/Qwen3.8-Flash-Next-oQ4e-mtp/tree/2322c888c2c314d4eceddf0159dec5d821aa1e41

Model (Qwen3.8-Flash-Next, built by the Qwen team)

huggingface.co/Qwen/Qwen3.8-Flash-Next

Engine

oMLX 0.7.3 — the program that loads the model and answers requests · github.com/jundot/omlx/releases/tag/v0.7.3

Context window

262,144 tokens: how much text the model can see at once

Steps

1. Get the engine and the model files from the list above.

2. Turn on MTP (multi-token prediction, depth 3) for this model: set mtp_enabled=true and mtp_fixed_depth=3 in model_settings.json in the base folder of your oMLX install (BASE_FOLDER in the command below) or on the admin page /admin. It is off by default (mtp_enabled=false).

3. Start the server with this command. It loads the model into memory. When the model has loaded, the server waits for requests on port 8888, only from this computer.

   Copy

   omlx serve \
     --model-dir MODELS_FOLDER \
     --base-path BASE_FOLDER \
     --host 127.0.0.1 \
     --port 8888 \
     --api-key YOUR_KEY

   Replace MODELS_FOLDER with the folder of your oMLX install that holds the model files, and BASE_FOLDER with the base folder of your oMLX install (the per-model settings file model_settings.json is in it). Replace YOUR_KEY with a key of your own.

   What the parts mean

   • --model-dir MODELS_FOLDERthe folder with the model files
   • --host 127.0.0.1who can connect: only this computer
   • --port 8888the network port the server listens on
   • --api-key YOUR_KEYa key that clients must send with every request; pick your own

oMLX 0.7.3 (MLX), MTP depth 3: model setting mtp_enabled=true + mtp_fixed_depth=3 (omlx/model_settings.py:493 (mtp_enabled), 502 (mtp_fixed_depth), 499 (mtp_adaptive_max_depth)), set per model in <base-path>/model_settings.json or admin /admin; default mtp_enabled=false; omlx serve --model-dir $HOME/engines/omlx-0.7.3/models --base-path $HOME/engines/omlx-0.7.3/base --host 127.0.0.1 --port 8888 --api-key speed2