niklaslenzMia's TensorFold recipe: Ornith-1.5-122B, EXL3 4 bpw
1x NVIDIA DGX Spark
Overall score as of 22 Oct 2026. Scores are relative to the best run in the class today; a better model later moves older runs down.
How the overall score is calculated
| Part | This run | Best in class | Relative | Weight | Points |
|---|---|---|---|---|---|
| Speed, one request | decode 42.3 tok/s | best 44.2 tok/s | 95.7 % | 22.5 % | 21.5 |
| Speed, 8 at once | total 60.0 tok/s | best 87.6 tok/s | 68.5 % | 22.5 % | 15.4 |
| Quality | mean of 7 tests: 69.0 % | best mean 70.8 % | 97.4 % | 30 % | 29.2 |
| Browser check | 8 of 10 | 80.0 % | 5 % | 4.0 | |
| My rating | 5 of 10 | 50.0 % | 20 % | 10.0 | |
| Overall | 100 % | 80.2 |
The seven quality tests: MMLU-Pro 58.5 · GPQA 49.0 · IFEval 80.5 · LiveCodeBench 45.0 · long text 8K 87.5 · long text 32K 82.5 · GPQA 30 with thinking 80.0.
The speed part is half the decode speed with one request and half the total speed with 8 requests at once, each relative to the best run of this class. Quality is relative to the best run of this class, the browser check and my rating are out of 10. The parts are rounded one by one. sparkDash, the memory test and speculative-decoding numbers are not part of it. How the score is made, and the assumption behind it.
What it builds as an agent
Task: Voxel world (3D scene, one HTML file). One attempt, no help. Counted: the final file counts.
The exact promptcopy it and try it yourself↓
This is the task prompt word for word, as we handed it to the agent harness. The harness adds its own system prompt and tools, and some harnesses add short context notes of their own. A run that goes past 90 minutes also gets a short “finish now” message, as the prompt says. To try it yourself, give it to your own coding agent: it needs a shell and a working folder.
Design and create the most impressive, creative, elaborate and detailed
voxel art world you are capable of. The theme and style are ENTIRELY your
choice — pick whatever world you can execute best and push it as far as
you can: scale, variety, detail, color, animation, atmosphere, life.
Surprise me.
Hard requirements:
- Everything ends up in ONE single self-contained HTML file (game.html in
the current working directory) that opens and runs directly in Chrome.
Use WebGL and whatever libraries you like, but embed everything —
no internet access at view time, no external files.
- The scene must present itself: include an automatic slow cinematic
camera tour (orbiting the world, occasionally zooming in on highlights)
that starts on load, plus mouse controls (orbit/zoom) for manual viewing.
- All user-facing text (UI, HUD, titles, descriptions, hints) must be in
ENGLISH, regardless of the language of any project files you encounter.
When finished, verify your HTML actually loads and renders without
console errors, then state DONE with the absolute path of game.html.
Do not ask questions.
How you work (same for every model):
- You work alone in the folder /work (this is the current working directory). Nobody will answer questions. Aim to finish in about 60 minutes. After 90 minutes you will get a message to finish; then you have at most 30 more minutes. Hard stop after 120 minutes.
- Tools on this machine: shell, files, python3 with numpy and Pillow, ffmpeg, node, and `shot.sh game.html` (opens the page in a headless browser, saves a screenshot and prints console errors). There is no internet and no access to other machines.
- Keep a working /work/game.html at all times. If you are stopped, the last version that loaded without errors counts.
- JavaScript libraries available offline in the test browser, through an import map with these exact URLs: "three": "https://cdn.jsdelivr.net/npm/three@0.170.0/build/three.module.js", "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.170.0/examples/jsm/", "simplex-noise": "https://cdn.jsdelivr.net/npm/simplex-noise@4.0.3/dist/esm/simplex-noise.js", "@tweenjs/tween.js": "https://cdn.jsdelivr.net/npm/@tweenjs/tween.js@25.0.0/dist/tween.esm.js", "cannon-es": "https://cdn.jsdelivr.net/npm/cannon-es@0.20.0/dist/cannon-es.js". No other external resources.
- After the scene is built, set `window.scene` to the Three.js scene (if you use Three.js) and `window.ready = true`.
Play it yourself
Open full screenAI-generated The page this setup built, running live in your browser: click into it and use the controls it shows itself (usually drag to look around, scroll to zoom). It needs WebGL and can be slow on a weak graphics card.
The browser check
- page loads without page error1 / 1
- window.ready === true within 30 s1 / 1
- no console errors (whole session)1 / 1
- frame is not blank1 / 1
- frame rate in software GL (rAF over 5 s)0 / 1
- world: scene size 0.5 (parts >= 80 or >= 20000 triangles; canvas fallback), visible text English if any 0.51 / 1
- automatic camera tour: the view changes on its own in two successive ~5 s windows (1 each)1 / 2
- animation in the world (nodes / instances / particles moving besides the camera)1 / 1
- OrbitControls: drag and wheel change the picture beyond the animation baseline1 / 1
Measured 2026-10-21. Raw files: run info · command · final choice · contract · sampling · gate stats · gate log · screenshot · check · clip.mp4 · web.mp4 · web-poster.jpg.
Browser check 8.0 out of 10: the page loads, runs without console errors, is not blank, shows a world (112,330 parts, 1,355,918 triangles, English text), animates (75 animated nodes) and answers to mouse drag and wheel; the 2 lost points are the frame rate (1.03 frames per second in software rendering without a GPU, 20 are needed) and the second window of the automatic camera tour (1 of 2 points: the picture changed in 1.5 % of its pixels, 5 % are needed). The check that ran together with the agent run failed at its screenshot step (timeout 30 s, no score; the measuring tool ended with exit code 79 = a part failed). The 8.0 comes from a second check of the same saved page file (sha256 384801bf...) with the screenshot timeout raised to 120 s; its files (check, screenshot, clip) are in the raw-file list above. The agent finished by itself after 23.5 minutes, one try, no help; the “finish now” message (from 90 minutes) was not needed. 10 versions of the page were seen, 1 loaded without errors, and that one counts. The limit per answer was 131,072 tokens, the value on the Ornith model card (README lines 175 and 187: max_new_tokens=131072). The recipe's server default is 32,768 tokens (MAX_TOKENS, README line 177, config.sh line 83) and applies only when a request sets none, so I raised the limit for this test; the server setup is unchanged. 43 requests, no error, no request shortened (1,976,683 prompt tokens, 44,012 answer tokens). Thinking was on with the recipe's sampling (temperature 0.6, top_p 0.95, top_k 20), the context was 262,144 tokens.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
run_showcase.sh --harness opencode --task VX --upstream http://127.0.0.1:37703/v1 --model Ornith-1.5-122B --sampling-file $RUN/sampling-show.json --out $RUN/raw/plan-2110-show-r2/show --limit-s 7200 --inject-after-s 5400 --poll-s 20 --ctx 262144How fast it runs
| Measurement | Result |
|---|---|
| Decode | 42.3 tok/s |
| Words per second | 29.7 words/s |
| Characters per second | 191.9 chars/s |
| Decode, code prompt | 43.5 tok/s |
| Decode, prose prompt | 41.1 tok/s |
| Spread of the decode runs (CV) | 5.9 % |
| Spread of the code runs (CV) | 5.9 % |
| Spread of the prose runs (CV) | 1.6 % |
| Decode, window 1 | 41.0 tok/s |
| Prefill at 2K | 2084.7 tok/s |
| Prefill at 8K | 2280.5 tok/s |
| Prefill at 32K | 1965.8 tok/s |
| Time to first token at 2K | 1.0 s |
| Time to first token at 8K | 3.6 s |
| Time to first token at 32K | 16.7 s |
| Draft acceptance rate | 42.0 % |
| Mean accepted length | 2.5 tokens |
| 1 request at once (total) | 41.4 tok/s |
| 2 at once (total) | 44.5 tok/s |
| 4 at once (total) | 59.1 tok/s |
| 8 at once (total) | 60.0 tok/s |
| 1 request at once (per request) | 41.4 tok/s |
| 2 at once (per request) | 23.0 tok/s |
| 4 at once (per request) | 14.9 tok/s |
| 8 at once (per request) | 10.7 tok/s |
| Server slots (from the recipe) | 4 slots |
| Long answer, wall time | 91.6 s |
Method: code+prose; 2 run(s) each; 512 tokens; 1 warm-up(s); median; requests at once (later supplementary run of 2026-10-22, same setup): C1,C2,C4,C8 at once; 2 rounds each; 512 tokens per request; prose prompt; total = all tokens / wall time, per request = tokens / end-to-end time; median over the rounds. Measured 2026-10-21 (requests at once: 2026-10-22). Raw files: speed result · speed contract · log · long answer · speed result · speed contract · log.
Requests at once: the server has 4 slots (from the author's recipe). C8 exceeds the slots: the extra requests wait in the queue, and they were measured anyway. The speed test ran with the recipe's own settings: four slots (README line 169), DFlash2 drafts on, fp8 KV cache, context 262,144 tokens, the request flag for no thinking and the recipe's sampling (temperature 0.6, top_p 0.95, top_k 20, min_p 0), seed 42. The headline 42.32 tok/s is the mean of the code prompt (43.52) and the prose prompt (41.11), medians of two runs each. The author's own table (README lines 33-45) lists 52.6 tok/s for prose and 83.3 tok/s for code with one stream, measured with sparkDash and thinking off; the README names sampling with drafts as not tested (line 227), so those numbers and ours, which run at temperature 0.6 with our own prompts, are not directly comparable. The values for 1, 2, 4 and 8 requests at once (total and per request) come from a later supplementary run on box-4, a second DGX Spark with the same recipe, image and checkpoint as box-3, which measured the base speed test; the recipe runs four slots, so the extra requests of C8 queue. The sparkDash section further down is a separate run with sparkDash's own prompts at temperature 0; it is not mixed with the speed test.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 speed2.py --base-url http://127.0.0.1:40559/v1 --label run-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/plan-2110-speed-r0/work/speed2 --contract $RUN/raw/plan-2110-speed-r0/work/speed2/speed-contract.json --parts decode,prefill,conc,l1 --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --tokenizer-json $HOME/.cache/huggingface/hub/models--Mia-AiLab--Ornith-1.5-122B-EXL3-EAQv1-4-bpw/snapshots/9c5c913b3de1eef901520d6fedaf450044ea0423/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.6, "top_p": 0.95, "top_k": 20, "min_p": 0.0} --sampling-source recipe README.md:176 (TEMPERATURE/TOP_P/TOP_K Default 0.6/0.95/20, 'from the model's generation_config.json') and the checkpoint's generation_config.json in HF snapshot 9c5c913b (GB10 box 2-Cache, temperature 0.6, top_p 0.95, top_k 20). The recipe does not name min_p -> 0.0. No separate thinking setting documented -> same values with thinking on/off. --model Ornith-1.5-122B --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt
# Requests at once: supplementary run of 2026-10-22 (raw folder speed-r0-conc)
python3 speed2.py --base-url http://127.0.0.1:38419/v1 --label run-speed-r0 --prompts $BENCH/repos/rigmark-v2/prompts-v2.json --output $RUN/raw/plan-2210-speed-r0/work/speed2 --contract $RUN/raw/plan-2210-speed-r0/work/speed2/speed-contract.json --parts conc --prefill-depths 2048,8192,32768 --prompt-ids code,prose --runs 2 --max-tokens 512 --warmups 1 --prefill-runs 3 --conc-levels 1,2,4,8 --conc-rounds 2 --conc-max-tokens 512 --conc-prompt prose --tokenizer-json $HOME/.cache/huggingface/hub/models--Mia-AiLab--Ornith-1.5-122B-EXL3-EAQv1-4-bpw/snapshots/9c5c913b3de1eef901520d6fedaf450044ea0423/tokenizer.json --tokenizer-python $HOME/not published --sampling {"temperature": 0.6, "top_p": 0.95, "top_k": 20, "min_p": 0.0} --sampling-source recipe README.md:176 (TEMPERATURE/TOP_P/TOP_K Default 0.6/0.95/20, 'from the model's generation_config.json') and the checkpoint's generation_config.json in HF snapshot 9c5c913b (GB10 box 2-Cache, temperature 0.6, top_p 0.95, top_k 20). The recipe does not name min_p -> 0.0. No separate thinking setting documented -> same values with thinking on/off. --model Ornith-1.5-122B --no-cache-prompt --server-log $RUN/logs/server-A.log --engine-version-file $RUN/logs/engine-version.txt --server-slots 4 --server-slots-source MiaAI-Lab/Ornith-1.5-122B-TensorFold@0c8fd6a0 scripts/config.sh:80 (PARALLEL=4, requests at once); README.md:169 (PARALLEL default 4) and :170 (CONTEXT 262144) --supplement-of $RUN/raw/plan-2110-speed-r0How well it answers
| Test | Result95 % margin | Right | Cut offcount as wrong | Raw files | |
|---|---|---|---|---|---|
| MMLU-Pro 400 fixed questions, letter only | 58.5 % ± 4.8 pts | 234 / 400 | 2 | answers · summary · log · contract | command |
| GPQA Diamond 198 questions, letter only | 49.0 % ± 6.9 pts | 97 / 198 | 0 | answers · summary · log · contract | command |
| IFEval 200 fixed prompts, strict | 80.5 % ± 5.5 pts | 161 / 200 | 0 | answers · summary · log · contract | command |
| LiveCodeBench 40 newest tasks, the code runs | 45.0 % ± 14.7 pts | 18 / 40 | 5 | summary · log · contract | command |
| Long text 8K 4 tasks x 10 at 8192 tokens | 87.5 % ± 10.3 pts | 35 / 40 | 0 | log · contract · answers · summary | command |
| Long text 32K 4 tasks x 10 at 32768 tokens | 82.5 % ± 11.6 pts | 33 / 40 | 0 | log · contract · answers · summary | command |
| GPQA 30, thinking on 30 questions, limit 16,384 tokens | 80.0 % ± 13.9 pts median 2,961 tokens, 63.8 s per question | 24 / 30 | 5 | log · contract · answers · summary | command |
Thinking off, except the last test. Seed 42. Sampling from the model card. A cut-off answer counts as wrong. Measured 2026-10-21.
Note from the run: Of the 400 MMLU-Pro answers 2 were cut off at the 48-token limit; of the 40 LiveCodeBench answers 5 at their limit of 32,768 tokens; of the 30 GPQA answers with thinking 5 at the limit of 16,384 tokens; and 3 of 285 sparkDash MMLU answers at its limit of 1,024 tokens. These count as wrong. Thinking was off in the instruct tests (the request flag was off in every request), yet the model sometimes writes its reasoning into the reply: one LiveCodeBench reply and one of the cut MMLU-Pro replies start with a think block. The official IFEval checker is not fully repeatable: seven re-runs on the stored answers gave 160 or 161 of 200 (80.0 or 80.5 %), inside the margin. GPQA Diamond ran on the public mirror fingertap/GPQA-Diamond (the original is gated). The tests of this page ran on two DGX Sparks with the same recipe, image and checkpoint: box-3 (speed, MMLU-Pro, GPQA, IFEval, memory, sparkDash) and box-4 (LiveCodeBench, RULER, GPQA with thinking, build test, supplementary concurrency run). These are my own tests of this pack.
Exact commands we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts mmlupro --job plan-2110bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts gpqa --job plan-2110bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts ifeval --job plan-2110bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts lcb --job plan-2110bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts ruler --job plan-2110bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts ruler --job plan-2110bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts think --job plan-2110How much context fits
| Context size | Prompttokens | Filled | First token | Prefilltok/s, as the server reports it | Decodetok/s | Peak memoryGiB | Result |
|---|---|---|---|---|---|---|---|
| 4K 4,096 tokens | 3,891 | 95.0 % | — | — | — | 76.45 | answered (length) |
| 8K 8,192 tokens | 7,782 | 95.0 % | — | — | — | 76.45 | answered (stop) |
| 16K 16,384 tokens | 15,564 | 95.0 % | — | — | — | 77.05 | answered (stop) |
| 32K 32,768 tokens | 31,129 | 95.0 % | — | — | — | 77.65 | answered (stop) |
| 64K 65,536 tokens | 62,259 | 95.0 % | — | — | — | 79.87 | answered (stop) |
| 128K 131,072 tokens | 124,518 | 95.0 % | — | — | — | 83.09 | answered (stop) |
| 256K 262,144 tokens | 249,036 | 95.0 % | — | — | — | 90.93 | answered (stop) |
Memory test. For each context size (4K, 8K, 16K ... up to the largest size tested for the setup) the server is started again with that context, one prompt that fills about 95 % of the context is sent, and the model must answer. The largest size that loaded and answered counts. Decode speed is measured on that answer at the largest size. Peak memory is the rise of used memory (MemTotal - MemAvailable) over the idle value before the step, on the one box. Unified memory, so GPU and CPU share it. Measured 2026-10-21. Raw files: result of every step · log · contract.
Note from the run: The largest size tested is 262,144 tokens, the recipe's own context (README line 170), so this is the largest size tested, not necessarily the largest the engine could load. Every size from 4K to 256K loaded and answered (a prompt filling 95 % of the window). The memory number is the rise over the idle value (unified memory) at the largest step and includes the model weights. At 64K and 256K the recipe's start script reported a failed smoke test (fit.json: start_rc 1 at 65536 and 262144), while the server stayed up and answered the test prompt of the step (finish reason stop); both sizes therefore count as loaded and answered. TensorFold returns no timing values, so there is no decode speed per step, and the sentence on decode speed in the method text does not apply to this run.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
bash tools/quality2/measure_all.sh tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json --parts fit --job plan-2110Also made this possible
- Model
Ornith AIHugging Face GitHub Website - Draft model (DFlash2)
Inco AI@inco_ai Hugging Face GitHub Website - Engine (TensorFold)
Ash Hart@ashxhart GitHub Hugging Face Website - EXL3 format
turboderp@turboderp_ GitHub Hugging Face
Run it yourself
What you need and how to start it, step by step. The results below come from exactly this setup.
What you need
- Hardware
- 1x NVIDIA DGX Spark
- Model
- Mia-AiLab/Ornith-1.5-122B-EXL3-EAQv1-4-bpw @9c5c913b (Mia's AI Lab; Qwen3.5-MoE architecture)
- Draft model (DFlash2)
- huggingface.co/incoai/Ornith-1.5-122B-DFlash2
- Engine
- TensorFold (Zig engine) with DFlash2 drafts on one DGX Spark — the program that loads the model and answers requests · github.com/ashhart/TensorFold
- Recipe and EXL3 EAQ v1 4 bpw checkpoint
- github.com/MiaAI-Lab/Ornith-1.5-122B-TensorFold by Mia (MiaAI-Lab)
- Context window
- 262,144 tokens: how much text the model can see at once
Steps
Get the recipe, the engine and the model files from the list above.
Start the server with this command. It loads the model into memory.
./start.sh ($HOME/ornith-2110/recipe @0c8fd6a0; = tensorfold-native serve with scripts/config.sh-Defaults: context 262144, parallel 4, KV fp8, DFlash2 drafter, thinking on, port 8888); via ornith-box.sh startThe file names are the ones we used. Run the command in the folder that holds them, or put the full path in front of each name.
Exact command we measured withfor reproducing↓
You do not need this to run the model. It is the start text exactly as our run recorded it, with the engine build in front, so anyone can repeat the result.
TensorFold Zig tensorfold-native @d1018b31 + Mia patches 0001-0027 (EXL3 for Qwen3.5-MoE), Image tensorfold-ornith:zig-d1018b3, NVIDIA-PyTorch-Container 26.07-py3; ./start.sh ($HOME/ornith-2110/recipe @0c8fd6a0; = tensorfold-native serve with scripts/config.sh-Defaults: context 262144, parallel 4, KV fp8, DFlash2 drafter, thinking on, port 8888); via $HOME/ornith-2110/ornith-box.sh startContext and sampling: 262144 (native window, recipe README:170), PARALLEL 4 (README:169), KV fp8 (recipe default, README:143), DFlash2 drafter on (README:173), thinking on as the server default (README:172; requests with thinking off via enable_thinking, README:189), Sampling 0.6/0.95/20 (README:176, from generation_config.json), seed 42. Hardware: 1x NVIDIA DGX Spark. All raw files of this setup: raw/v2/ornith-15-122b-exl3-1spark-tensorfold.
Mia's own benchmark, run unchanged
This is Mia's own benchmark tool (commit 21a3511), run unchanged. It has its own section on this page, and it is never mixed with my numbers. A cut-off answer counts as wrong here, so a value can differ from sparkDash's own screen.
| Test | Result95 % margin | Right | Cut offcount as wrong | Raw files | |
|---|---|---|---|---|---|
| Quality QA 150 questions | 86.0 % ± 5.6 pts | 129 / 150 | 0 | summary | command |
| Quality MMLU 285 questions | 86.7 % ± 4.0 pts | 247 / 285 | 3 | summary | command |
| Tool Eval Quick 15 scenarios, tool use | 93 / 100 | 13 / 15 passed | — | tool eval result · tool eval report · tool eval command · summary | command |
| Decode, 1 request at once first token 158 ms | 152.0 tok/s | — | — | decode result · summary | command |
| Prefill at 4K first token 1.9 s | 2446.3 tok/s | — | — | prefill result · summary | command |
| Prefill at 8K first token 3.8 s | 2271.0 tok/s | — | — | prefill result · summary | command |
| Prefill at 16K first token 7.8 s | 2156.9 tok/s | — | — | prefill result · summary | command |
| Prefill at 32K first token 17.0 s | 1957.6 tok/s | — | — | prefill result · summary | command |
Seed 1234 applies to the two quality tests; decode, prefill and tool eval send no seed. The benchmark client ran on a separate machine that is never part of the measured group. Measured 2026-10-21. Version 2.50.0, commit 21a3511.
QA. Quality QA, 150 items, seed 1234, temperature 0, thinking off, 4 requests at once (Mia's default). An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).
MMLU. Quality MMLU, 285 items (5 for each of 57 subjects, 4 answers), seed 1234, temperature 0, thinking off. An answer cut off at the token limit counts as wrong. ± is the 95 % margin (Wilson).
Decode. Decode bench, 1 request at once, 400 tokens, prompt type structured, temperature 0, thinking off. No seed is sent.
Prefill. Prefill bench at 4K, 8K, 16K and 32K tokens. sparkDash skips a size that does not fit into the context. No seed is sent.
Tool Eval Quick. Tool Eval Quick: tool-eval-bench (SeraphimSerapis, commit c61319b), 15 core scenarios (--short), --no-think, --timeout 120. Score 0 to 100. No seed is sent.
Note from the run: sparkDash 2.50.0 (@21a3511) on a client machine (never in the measured group), Quality: seed 1234, temperature 0, thinking off, a truncated answer counts as wrong; decode, prefill and tool eval send no seed; own row, never mixed with quality-v2.
Exact commands we measured withfor reproducing↓
You do not need this to run the model. It is the exact command of our measurement, so anyone can repeat the result.
python3 tools/quality2/sparkdash_client.py tools/quality2/cards/ornith15-122b-exl3-4bpw-tensorfold-1x.json <box> --client <client machine> --job plan-2110 --out $RUNtool-eval-bench --base-url http://127.0.0.1:<port> --model Ornith-1.5-122B --short --no-think --timeout 120 --json-file $BENCH/sparkdash-client/runs/work/tooleval/result.json --no-live