Ash's TensorFold recipe: LFM2.5-24B-A2B (MLX 4-bit) on a Mac Studio

How fast it runs

Long prompts

Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; context steps: prompts of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last), 2 runs per step, 512 output tokens; decode = tokens after the first / time after the first token; prefill = prompt tokens / time to first token of the same request, median over its requests; requests at once: C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-11-10. Raw files: speed result · speed contract · log.

I ran Ash Hart's recipe for this model (docs/recipes/lfm2.5-24b-a2b.md at main, read on 10 Nov 2026, lines 3 to 11): tensorfold-native serve <model folder> --port 8080 --parallel 16. The weights are Liquid AI's own MLX 4-bit export, LiquidAI/LFM2.5-24B-A2B-MLX-4bit, at the revision the recipe names (667408d4): MLX affine 4 bits, group size 64, token embedding at 6 bits, converted by Liquid AI with mlx-lm 0.32.0 according to its model card, 13.6 GB in 3 weight files. All 3 weight files and the tokenizer match the sha256 listed by Hugging Face. The engine is the official TensorFold 1.2.5 release archive for macOS (tensorfold-1.2.5-macos-arm64, sha256 563147f1; its binary tensorfold-native has the sha256 13ce9fcc), not built by me.

Differences from the author's recipe, all named: the recipe builds the engine from source (zig build, lines 9 and 10); I used the official release archive instead. I added --host 127.0.0.1 and --no-update-check (the README calls it an accepted compatibility switch, line 364), --keep-warm 86400 (keeps the Metal GPU active while idle for a day; the README default is 900 seconds, line 326) and I used --port 8888 where the recipe writes 8080. I set no --context, so the checkpoint's window of 131,072 tokens applies, as in the recipe. The recipe's own table was measured on main at commit 0d24ecb3 plus the shared-round change (its line 32); the changelog lists that change under release 1.1.8 (line 1375 of my copy), so release 1.2.5 contains it. Two earlier starts of my tests (a card check that stopped before anything was measured, and a card with a wrong tokenizer path) produced no numbers; nothing of them is on this page.

The headline 251.5 tok/s is the mean of the code prompt (226.7) and the prose prompt (276.4), each the median of two runs (spread 0.5 % and 12.2 %). The spread of the prose runs (252.6 and 300.1 tok/s) is below my limit of 15 % for repeating the test, but it is well above that of the code runs, so I show it here. In the context steps the two prose runs of 32K, 64K and 130,048 tokens differ by more than 15 % (spread 16.5 %, 16.9 % and 22.5 %; the tool flags them as noisy); they are shown as measured. Thinking was off in every request (the model thinks by default): none of the 50 requests listed in the speed result file shows reasoning text (reasoning_chars 0). The tool's cache check found no cached prompt tokens in any of the 60 requests it examined. No draft model was used. The server's own copy drafts were accepted at a rate of 58.9 % (274 of 465 drafted tokens), with a mean accepted length of 1.22 tokens. I measured with sampling (temperature 0.1, top_k 50, seed 42) and 512 answer tokens per request. repetition_penalty 1.05 (Liquid) not sent, server default: Liquid AI recommends that value (its generation_config.json), but my test tools send no such field, so no request carried it. The server's start line names only temperature 0.1 and top_k 50, so I do not know whether the server applies a repetition penalty on its own.

Requests at once: the server has 16 slots (--parallel 16, the recipe's setting), so 1, 2, 4 and 8 requests all fit; 16 at once were not measured. The total grows from 230.9 tok/s for 1 request to 639.1 tok/s for 8 requests; the speed of one request falls from 230.9 to 85.3 tok/s. The total at 1 request is lower than the decode headline because that test counts the whole answer time, including the wait for the first token, and uses the prose prompt only.

The context steps run a prompt of exactly 16K, 32K, 64K and 128K tokens (a filler log first, the code or prose task prompt last) with 512 answer tokens. The 128K step is a prompt of 130,048 tokens, not 131,072: a prompt of 131,072 tokens plus 512 answer tokens does not fit into the server's window of 131,072, so the tool uses the window minus 1,024 (its rule of 29 Oct 2026). The headline prefill figures are for 2K, 8K and 32K prompts.

The recipe page has its own speed table (lines 29 to 40): greedy, 256 tokens, 16 public prompts, one per stream, 3 repeats in each of 3 alternated rounds on an M5 Ultra, aggregate tokens per second over all streams: 266.5 to 267.7 for 1 stream, 565.2 to 566.1 for 4, 732.2 to 735.6 for 8 and 922.5 to 923.5 for 16 (the page notes that this is main at 0d24ecb3 plus the shared-round change). My tests are different (sampled, 512 tokens, a code and a prose prompt, the median of each, and for the requests at once the total of all requests), so the two are not like for like and I do not call any difference a deviation, and I have not looked for a cause. For the record: my total at 1 request at once is 230.9 tok/s, at 4 requests 480.8 and at 8 requests 639.1; my prose decode is 276.4 tok/s. I have not repeated the author's test with the author's prompts.

The Mac: it was my workstation and ran other things during the tests. The test's load guard (it repeats a step when one other process stays above 25 % of one core for two samples in a row; a sample every 0.5 seconds) did not count three kinds of processes as foreign load, and they ran on the same Mac during all tests: the Claude Code session that controlled the tests (a command-line assistant that started the test tools and the server, with its search helpers), the macOS desktop shell (window server, Dock, Finder, menu bar and notification services) and my voice-dictation helper. The guard repeated 1 of its 37 steps (the 32K context step, prose run 1): a short ffmpeg job of the dictation helper was running, the guard threw that try away and ran the step again after 5.2 seconds of quiet. In 10 of the 37 steps one macOS system service reached a single sample above 25 % of one core (up to 33.6 %); none stayed above for two samples in a row, so the guard repeated none of them. The quiet check (a sample every 10 seconds) saw at most 17.9 % of one core in 24 samples. My script paused three macOS background services (corespotlightd, mediaanalysisd, photoanalysisd) before the run and resumed them afterwards. So the speed numbers were not measured on an otherwise idle Mac.

python3 speed2.py --base-url http://127.0.0.1:52328/v1 --label mac256-run-lfm-s-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $HOME/lfm25-24b/out/mac256-run-lfm-s/raw/ --contract $HOME/lfm25-24b/out/mac256-run-lfm-s/raw/ --parts decode,prefill,conc,ctx --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --ctx-depths 16384,32768,65536,131072 --ctx-runs 2 --ctx-max-tokens 512 --tokenizer-json $HOME/models/lfm25-24b/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.1, "top_k": 50} --sampling-source LiquidAI/LFM2.5-24B-A2B-MLX-4bit @667408d4 generation_config.json (temperature 0.1, top_k 50, repetition_penalty 1.05, do_sample true) = the 'recommended generation settings' of the model card LiquidAI/LFM2.5-24B-A2B (temperature 0.1, top_k 50, repetition_penalty 1.05). No top_p or min_p is named, so the server defaults apply. Seed 42 for speed and quality. The model is a pure reasoning model: its chat template opens <think> itself. My tools switch thinking off per request (enable_thinking false), except in GPQA 30. repetition_penalty 1.05 (Liquid) not sent, server default --model lfm2.5-24b-a2b --no-cache-prompt --server-log $HOME/lfm25-24b/logs/server.log --engine-version-file $HOME/lfm25-24b/out/mac256-run-lfm-s/logs/engine-version.txt --server-slots 16 --server-slots-source docs/recipes/lfm2.5-24b-a2b.md (ashhart/TensorFold main) Run: 'serve <model> --port 8080 --parallel 16'; the server names how many streams fit at the 131,072 window (read from the start line). C8 fills 8 of the 16 slots. --server-ctx 131072 --server-ctx-source Recipe (LiquidAI/LFM2.5-24B-A2B-MLX-4bit config: 131,072 window; recipe sets no --context, default = checkpoint window) and README 'Context length: 131,072 tokens'

How well it answers

All seven quality tests ran. Six ran with thinking off and one (GPQA 30) with thinking on; Liquid AI describes the model as a pure reasoning model, and it is built for agents rather than for knowledge-heavy tasks, which these tests do not measure. The notes below say how the numbers were taken.

Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-11-10.

Note from the run: MMLU-Pro and GPQA Diamond ran with an answer limit of 1,024 tokens instead of my usual 48 (the setting of this setup's test card, as on my other Mac setups); same questions, seed 42, the first letter of the answer counts. IFEval allowed 4,096 tokens per answer, long text 128, LiveCodeBench 32,768 and GPQA 30 with thinking 16,384. 16 questions ran at a time, the stream count of the server. LiveCodeBench ran in its own run (a separate start of the same server command) after the other tests; its code sandbox runs in a container VM that I started for that run and stopped afterwards. GPQA Diamond ran from the public mirror fingertap/GPQA-Diamond (198 questions), because the original is gated and no token was available.

Cut-off answers count as wrong: 9 in MMLU-Pro, 12 in GPQA Diamond, 4 in IFEval, 8 in long text (3 at 8K, 5 at 32K), 0 in LiveCodeBench and 12 of the 30 GPQA answers with thinking (limit 16,384 tokens). Without LiveCodeBench and the thinking test that is 33 of 878 answers. The median answer of the thinking test used 11,196 tokens and 204.2 seconds, with 16 questions running at a time.

Liquid AI calls LFM2.5-24B-A2B a pure reasoning model: its chat template opens a thinking block itself, and its model card says it always thinks before it answers. In the TensorFold server the request field enable_thinking false switches that off (no reasoning text in the requests of the speed test), so the six tests with thinking off run outside the mode the model was designed for, and I have not measured how much that changes the numbers. The median MMLU-Pro answer is 3 tokens long (the letter). Only GPQA 30 ran with thinking on. The model card also says that the model is built for agents that reason and call tools and is not recommended for knowledge-heavy tasks on its own; my tests (general knowledge, science questions, instruction following, coding, long text) do not measure that strength. repetition_penalty 1.05 (Liquid) not sent, server default: Liquid AI recommends it, my test tools send no such field, so no request of any test carried it.

LiveCodeBench: 1 of the 40 answers held no code; the median answer used 997 tokens, none hit the limit of 32,768. The recipe page gives no quality numbers for this checkpoint, so these are my own tests. Two answers files are not published, because the site's filter for machine names and power readings held them back: the IFEval answers (one answer names a made-up company, and the filter reads a word in that name as the name of a machine) and the LiveCodeBench answers (a program in an answer has a variable named power that computes powers of two; it is not a power reading). The summaries, the logs and the contracts of both tests are published, and the numbers of the table were counted from the complete files.

Foreign load: the quality tests have no guard; their quiet check records the busiest other process every 10 seconds. The Claude Code session that controlled the tests and the macOS desktop shell ran on the same Mac during all of them (see the speed note). Samples above 25 % of one core: in the instruction tests 1 of 35 (28.2 %, a macOS system service); in the thinking test 4 of 41 (a Python process at 35.4 %, the Claude Code session at 31.5 % and a macOS audio service at 31.2 % and 27.1 %); in the LiveCodeBench run 2 of 28, both at 100 % of one core and the last two of the run: the container VM of the code sandbox of that test itself, while the answers' code was run. The three macOS background services named in the speed note were paused for each run and resumed afterwards.

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts mmlupro --job mac256-run-lfm-q

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts gpqa --job mac256-run-lfm-q

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts ifeval --job mac256-run-lfm-q

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts lcb --job mac256-run-lfm-lcb

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts ruler --job mac256-run-lfm-q

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts ruler --job mac256-run-lfm-q

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts think --job mac256-run-lfm-q

How much context fits

Memory test. The recipe fixes the context (131,072 tokens), so the server is started once with the recipe's own command and reserves its memory up front. Then, for each size (4K, 8K, 16K ... up to the largest size tested), one prompt that fills about 95 % of that size is sent, and the model must answer. The largest size that answered counts; it is the highest size tested, not a limit of the setup. Decode speed at that size was not measured. Peak memory is the rise of used memory over the idle value before each server start (unified memory, so GPU and CPU share it). Measured 2026-11-10. Raw files: result of every step · log · contract.

Note from the run: fit memory per node, highest rise over idle: local 25.65 GiB. The server was started once, with the checkpoint's window of 131,072 tokens and 16 streams (the recipe's setting; the window is the model's own, not one I chose), and for each size one prompt that fills about 95 % of it was sent (prompts of 3,891 to 124,518 tokens, 96 answer tokens at most); the model answered at every size, so 131,072 is the largest size tested and the window of this setup, not a limit that was hit. The memory number is the rise of used memory over the idle value of the Mac before the server was started (unified memory), with that prompt in flight: 14.39 GiB at 4K, 16.74 GiB at 32K and 25.65 GiB at 131,072 tokens (the loading of the weights alone peaked at 14,366 MiB). The decode speed per size was not recorded: the server gives the test tool no timings, so the table shows no decode value.

bash tools/quality2/measure_all.sh tools/quality2/cards/lfm2.5-24b-a2b-mlx4bit-tf-1.2.5-official-m5ultra-1010.json --parts fit --job mac256-run-lfm-fit

How it plays the Gauntlet

One solo race, season seed 21, one attempt, no help.

Game rank as of Nov 10, 2026. The rank is the order of the Gauntlet finish times in this memory class. Race page and raw files.

The race ran the recipe named on its race page, with the sampling and answer limit of its lane config. The engine build of the race is not recorded in the race files.

Result (result.json) Race log (log.jsonl) Model answers (io.jsonl) System prompt Exact command Verify output Model config Race page

Also made this possible

• Model (LFM2.5-24B-A2B)Liquid AI@liquidai Hugging Face GitHub Website
• Model files (MLX 4-bit export, revision 667408d4)Liquid AI@liquidai Hugging Face GitHub Website
• Engine (TensorFold 1.2.5, official release archive for macOS arm64)Ash HartGitHub @ashxhart Hugging Face Website

Run it yourself

What you need and how to start it, step by step. The results below come from exactly this setup.

What you need

Hardware

Mac Studio M5 Ultra, 256 GB unified memory, one box (Darwin kernel 27.0.0)

System

macOS (the Mac above)

Model

LiquidAI LFM2.5-24B-A2B (hybrid short-conv + attention MoE, 24B total / 2B active, pure reasoning model), LiquidAI/LFM2.5-24B-A2B-MLX-4bit @667408d4 (MLX affine 4-bit, group 64, token embedding 6-bit, router gates fp32; official Liquid AI export)

Model (LFM2.5-24B-A2B)

huggingface.co/LiquidAI/LFM2.5-24B-A2B

Model files (MLX 4-bit export, revision 667408d4)

huggingface.co/LiquidAI/LFM2.5-24B-A2B-MLX-4bit/tree/667408d494bde191c2cb9dd4f4549827816b9275

Engine

TensorFold 1.2.5, the official macOS release (native Metal engine), Liquid AI's own MLX 4-bit weights, no draft model, the checkpoint's window of 131,072 tokens, 16 streams — the program that loads the model and answers requests · github.com/ashhart/TensorFold/releases/tag/v1.2.5

Recipe (docs/recipes/lfm2.5-24b-a2b.md, main) and engine (TensorFold 1.2.5, official release archive for macOS arm64)

github.com/ashhart/TensorFold/blob/main/docs/recipes/lfm2.5-24b-a2b.md by Ash Hart

Steps

1. Get the recipe, the engine and the model files from the list above.

TensorFold 1.2.5 official release archive macos-arm64 (tensorfold-native, tag v1.2.5), native Metal lfm2_moe family, default window 131072, --parallel 16, no draft model, text only; tensorfold-native serve --host 127.0.0.1 --port 8888 --parallel 16 --keep-warm 86400 --no-update-check <folder of LiquidAI/LFM2.5-24B-A2B-MLX-4bit @667408d4> (Ash's recipe docs/recipes/lfm2.5-24b-a2b.md: tensorfold-native serve <model folder> --port 8080 --parallel 16; deviations on this page: --host 127.0.0.1, --port 8888, --no-update-check, --keep-warm 86400)