eval-harness

dataset

orrery_triage_v1_labelled

65 items · 336 arms. Headline: each arm's own primary metric, mean over its items; arms with different primary metrics are not ranked against each other.

baseline, promoted 2026-10-31

triage_baseline_pro_v1

Nothing is decided at n = 65: on the combined cost the paired test finds no arm under the false-alarm ceiling significantly different from the leader, the arm that always dismisses among them; the ceiling (production's 4-pass mean, 21.25 of 31) was set after the results and sits within one or two false alarms of 18 rows; a no-decision scored as an escalation, a path production does not have. Production's own model is promoted as the baseline the next round measures against (owner, 2026-10-31). Next: a no-decision counts as wrong on both sides, the ceiling becomes no worse than this baseline by the paired test, and the gold grows to the 93 owner-labelled decisions before machines are booked.

Inspect the 317 arms on triage_cost, scored against gold/orrery_triage_v1_labelled (the first 40 of 317; add others in the inspector)Inspect the 19 arms on false_dismiss, scored against gold/orrery_triage_v1_labelledOpen its sheet

goals: Triage decisions on Orrery's error signals

01arms

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

336 shown
armmetricmeanitemscost per itemspeed per itemrunsscored against
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1
served on dgx
triage_cost0.400 ↓65$0.000000 · 4,813 tokp50 7,347 · p95 11,042 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd
served on dgx
triage_cost0.400 ↓65$0.000000 · 4,813 tokp50 7,347 · p95 11,042 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c
served on dgx
triage_cost0.400 ↓65$0.000000 · 4,813 tokp50 7,347 · p95 11,042 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b
served on dgx
triage_cost0.400 ↓65$0.000000 · 4,813 tokp50 7,347 · p95 11,042 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f
served on dgx
triage_cost0.400 ↓65$0.000000 · 4,813 tokp50 7,347 · p95 11,042 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v1
served on x3
triage_cost0.400 ↓65$0.000000 · 3,619 tokp50 16,207 · p95 29,798 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f
served on x3
triage_cost0.400 ↓65$0.000000 · 3,619 tokp50 16,207 · p95 29,798 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama31_8b_v1
served on dgx
triage_cost0.415 ↓65$0.000000 · 5,545 tokp50 5,179 · p95 7,879 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd
served on dgx
triage_cost0.415 ↓65$0.000000 · 5,545 tokp50 5,179 · p95 7,879 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.415 ↓65$0.000000 · 5,545 tokp50 5,179 · p95 7,879 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b
served on dgx
triage_cost0.415 ↓65$0.000000 · 5,545 tokp50 5,179 · p95 7,879 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.415 ↓65$0.000000 · 5,545 tokp50 5,179 · p95 7,879 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_llama31_8b_v1
served on mac
triage_cost0.415 ↓65$0.000000 · 5,565 tokp50 7,125 · p95 10,541 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd
served on mac
triage_cost0.415 ↓65$0.000000 · 5,565 tokp50 7,125 · p95 10,541 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c
served on mac
triage_cost0.415 ↓65$0.000000 · 5,565 tokp50 7,125 · p95 10,541 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_llama31_8b_v1 · rescored 77adc97b
served on mac
triage_cost0.415 ↓65$0.000000 · 5,565 tokp50 7,125 · p95 10,541 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f
served on mac
triage_cost0.415 ↓65$0.000000 · 5,565 tokp50 7,125 · p95 10,541 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd
served on x3
triage_cost0.415 ↓65$0.000000 · 5,490 tokp50 5,389 · p95 8,370 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c
served on x3
triage_cost0.415 ↓65$0.000000 · 5,490 tokp50 5,389 · p95 8,370 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_llama31_8b_v1 · rescored 77adc97b
served on x3
triage_cost0.415 ↓65$0.000000 · 5,490 tokp50 5,389 · p95 8,370 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f
served on x3
triage_cost0.415 ↓65$0.000000 · 5,490 tokp50 5,389 · p95 8,370 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v1
served on dgx
triage_cost0.477 ↓65$0.000000 · 3,538 tokp50 12,488 · p95 21,540 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.477 ↓65$0.000000 · 3,538 tokp50 12,488 · p95 21,540 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_escalate_v1
run on Linux/x86_64, 32 CPUs
triage_cost0.477 ↓651,100 tokp50 0.085 · p95 0.13 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_escalate_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.477 ↓651,100 tokp50 0.085 · p95 0.13 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_escalate_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.477 ↓651,100 tokp50 0.085 · p95 0.13 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_escalate_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.477 ↓651,100 tokp50 0.085 · p95 0.13 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_escalate_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.477 ↓651,100 tokp50 0.085 · p95 0.13 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v1
served on dgx
triage_cost0.523 ↓65$0.000000 · 6,295 tokp50 14,013 · p95 22,893 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.523 ↓65$0.000000 · 6,295 tokp50 14,013 · p95 22,893 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b
served on dgx
triage_cost0.523 ↓65$0.000000 · 6,295 tokp50 14,013 · p95 22,893 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.523 ↓65$0.000000 · 6,295 tokp50 14,013 · p95 22,893 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v1
served on x3
triage_cost0.523 ↓65$0.000000 · 7,352 tokp50 33,430 · p95 80,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c
served on x3
triage_cost0.523 ↓65$0.000000 · 7,352 tokp50 33,430 · p95 80,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b
served on x3
triage_cost0.523 ↓65$0.000000 · 7,352 tokp50 33,430 · p95 80,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f
served on x3
triage_cost0.523 ↓65$0.000000 · 7,352 tokp50 33,430 · p95 80,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.538 ↓65$0.001368 · 3,125 tokp50 3,353 · p95 5,695 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.538 ↓65$0.001368 · 3,125 tokp50 3,353 · p95 5,695 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_35_flash_lite_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.538 ↓65$0.001368 · 3,125 tokp50 3,353 · p95 5,695 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.538 ↓65$0.001368 · 3,125 tokp50 3,353 · p95 5,695 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.554 ↓65$0.000668 · 5,346 tokp50 8,324 · p95 11,239 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.554 ↓65$0.000668 · 5,346 tokp50 8,324 · p95 11,239 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.554 ↓65$0.000668 · 5,346 tokp50 8,324 · p95 11,239 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v1
served on mac
triage_cost0.554 ↓65$0.000000 · 3,429 tokp50 17,421 · p95 30,731 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f
served on mac
triage_cost0.554 ↓65$0.000000 · 3,429 tokp50 17,421 · p95 30,731 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v2
served on x3
triage_cost0.554 ↓65$0.000000 · 5,306 tokp50 5,534 · p95 7,717 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd
served on x3
triage_cost0.554 ↓65$0.000000 · 5,306 tokp50 5,534 · p95 7,717 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c
served on x3
triage_cost0.554 ↓65$0.000000 · 5,306 tokp50 5,534 · p95 7,717 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.010918 · 4,160 tokp50 11,696 · p95 18,138 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.010918 · 4,160 tokp50 11,696 · p95 18,138 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.010918 · 4,160 tokp50 11,696 · p95 18,138 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.010918 · 4,160 tokp50 11,696 · p95 18,138 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000715 · 4,764 tokp50 11,518 · p95 21,428 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000715 · 4,764 tokp50 11,518 · p95 21,428 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000715 · 4,764 tokp50 11,518 · p95 21,428 ms1gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000998 · 5,684 tokp50 6,867 · p95 13,887 ms1gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000998 · 5,684 tokp50 6,867 · p95 13,887 ms1gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000998 · 5,684 tokp50 6,867 · p95 13,887 ms1gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.585 ↓65$0.000998 · 5,684 tokp50 6,867 · p95 13,887 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_12b_noreason_v1
served on dgx
triage_cost0.585 ↓65$0.000000 · 3,389 tokp50 5,603 · p95 8,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd
served on dgx
triage_cost0.585 ↓65$0.000000 · 3,389 tokp50 5,603 · p95 8,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c
served on dgx
triage_cost0.585 ↓65$0.000000 · 3,389 tokp50 5,603 · p95 8,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b
served on dgx
triage_cost0.585 ↓65$0.000000 · 3,389 tokp50 5,603 · p95 8,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f
served on dgx
triage_cost0.585 ↓65$0.000000 · 3,389 tokp50 5,603 · p95 8,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_12b_noreason_v1
served on mac
triage_cost0.585 ↓65$0.000000 · 3,311 tokp50 8,599 · p95 13,904 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd
served on mac
triage_cost0.585 ↓65$0.000000 · 3,311 tokp50 8,599 · p95 13,904 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c
served on mac
triage_cost0.585 ↓65$0.000000 · 3,311 tokp50 8,599 · p95 13,904 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b
served on mac
triage_cost0.585 ↓65$0.000000 · 3,311 tokp50 8,599 · p95 13,904 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f
served on mac
triage_cost0.585 ↓65$0.000000 · 3,311 tokp50 8,599 · p95 13,904 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v2
served on mac
triage_cost0.585 ↓65$0.000000 · 5,696 tokp50 5,737 · p95 7,706 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd
served on mac
triage_cost0.585 ↓65$0.000000 · 5,696 tokp50 5,737 · p95 7,706 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c
served on mac
triage_cost0.585 ↓65$0.000000 · 5,696 tokp50 5,737 · p95 7,706 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 21,363 · p95 33,502 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 20,275 · p95 32,697 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 21,363 · p95 33,502 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 20,275 · p95 32,697 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 21,363 · p95 33,502 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 20,275 · p95 32,697 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 21,363 · p95 33,502 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats
served on x3
triage_cost0.600 ↓65$0.000000 · 6,171 tokp50 20,275 · p95 32,697 ms1gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.610 ↓65$0.001024 · 5,815 tokp50 7,273 · p95 12,605 ms3gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.610 ↓65$0.001024 · 5,815 tokp50 7,273 · p95 12,605 ms3gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored 77adc97b, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.610 ↓65$0.001024 · 5,815 tokp50 7,273 · p95 12,605 ms3gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.610 ↓65$0.001024 · 5,815 tokp50 7,273 · p95 12,605 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.626 ↓65$0.000684 · 4,621 tokp50 10,962 · p95 18,893 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.626 ↓65$0.000684 · 4,621 tokp50 10,962 · p95 18,893 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.626 ↓65$0.000684 · 4,621 tokp50 10,962 · p95 18,893 ms3gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.631 ↓65$0.000627 · 3,330 tokp50 6,448 · p95 11,349 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.631 ↓65$0.000627 · 3,330 tokp50 6,448 · p95 11,349 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.631 ↓65$0.000627 · 3,330 tokp50 6,448 · p95 11,349 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_4_argon_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.013617 · 4,751 tokp50 23,508 · p95 61,827 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_4_argon_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.013617 · 4,751 tokp50 23,508 · p95 61,827 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_4_argon_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.013617 · 4,751 tokp50 23,508 · p95 61,827 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_4_argon_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.013617 · 4,751 tokp50 23,508 · p95 61,827 ms1gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.019130 · 5,576 tokp50 43,158 · p95 79,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.019130 · 5,576 tokp50 43,158 · p95 79,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.646 ↓65$0.019130 · 5,576 tokp50 43,158 · p95 79,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v2
served on dgx
triage_cost0.646 ↓65$0.000000 · 5,506 tokp50 3,850 · p95 5,195 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd
served on dgx
triage_cost0.646 ↓65$0.000000 · 5,506 tokp50 3,850 · p95 5,195 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c
served on dgx
triage_cost0.646 ↓65$0.000000 · 5,506 tokp50 3,850 · p95 5,195 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v1
served on dgx
triage_cost0.646 ↓65$0.000000 · 7,756 tokp50 34,999 · p95 73,667 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.646 ↓65$0.000000 · 7,756 tokp50 34,999 · p95 73,667 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b
served on dgx
triage_cost0.646 ↓65$0.000000 · 7,756 tokp50 34,999 · p95 73,667 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.646 ↓65$0.000000 · 7,756 tokp50 34,999 · p95 73,667 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v1
served on mac
triage_cost0.646 ↓65$0.000000 · 7,484 tokp50 55,843 · p95 129,188 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c
served on mac
triage_cost0.646 ↓65$0.000000 · 7,484 tokp50 55,843 · p95 129,188 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b
served on mac
triage_cost0.646 ↓65$0.000000 · 7,484 tokp50 55,843 · p95 129,188 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f
served on mac
triage_cost0.646 ↓65$0.000000 · 7,484 tokp50 55,843 · p95 129,188 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.662 ↓65$0.010847 · 4,156 tokp50 12,311 · p95 19,302 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.662 ↓65$0.010847 · 4,156 tokp50 12,311 · p95 19,302 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.662 ↓65$0.010847 · 4,156 tokp50 12,311 · p95 19,302 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.662 ↓65$0.010847 · 4,156 tokp50 12,311 · p95 19,302 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v2
served on dgx
triage_cost0.662 ↓65$0.000000 · 3,135 tokp50 7,719 · p95 10,098 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd
served on dgx
triage_cost0.662 ↓65$0.000000 · 3,135 tokp50 7,719 · p95 10,098 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c
served on dgx
triage_cost0.662 ↓65$0.000000 · 3,135 tokp50 7,719 · p95 10,098 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_phi4_14b_v1
served on dgx
triage_cost0.662 ↓65$0.000000 · 6,064 tokp50 17,140 · p95 23,284 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd
served on dgx
triage_cost0.662 ↓65$0.000000 · 6,064 tokp50 17,140 · p95 23,284 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.662 ↓65$0.000000 · 6,064 tokp50 17,140 · p95 23,284 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b
served on dgx
triage_cost0.662 ↓65$0.000000 · 6,064 tokp50 17,140 · p95 23,284 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.662 ↓65$0.000000 · 6,064 tokp50 17,140 · p95 23,284 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.677 ↓65$0.000849 · 3,854 tokp50 16,361 · p95 31,807 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.677 ↓65$0.000849 · 3,854 tokp50 16,361 · p95 31,807 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.677 ↓65$0.000849 · 3,854 tokp50 16,361 · p95 31,807 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v1
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,384 tokp50 46,681 · p95 127,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,384 tokp50 46,681 · p95 127,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,384 tokp50 46,681 · p95 127,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,384 tokp50 46,681 · p95 127,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v2
served on dgx
triage_cost0.677 ↓65$0.000000 · 6,286 tokp50 6,762 · p95 8,238 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd
served on dgx
triage_cost0.677 ↓65$0.000000 · 6,286 tokp50 6,762 · p95 8,238 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c
served on dgx
triage_cost0.677 ↓65$0.000000 · 6,286 tokp50 6,762 · p95 8,238 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v1
served on dgx
triage_cost0.677 ↓65$0.000000 · 4,514 tokp50 36,513 · p95 70,263 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.677 ↓65$0.000000 · 4,514 tokp50 36,513 · p95 70,263 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b
served on dgx
triage_cost0.677 ↓65$0.000000 · 4,514 tokp50 36,513 · p95 70,263 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.677 ↓65$0.000000 · 4,514 tokp50 36,513 · p95 70,263 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v2
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,246 tokp50 15,795 · p95 21,883 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,246 tokp50 15,795 · p95 21,883 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c
served on dgx
triage_cost0.677 ↓65$0.000000 · 5,246 tokp50 15,795 · p95 21,883 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v1
served on mac
triage_cost0.677 ↓65$0.000000 · 5,271 tokp50 91,096 · p95 182,205 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c
served on mac
triage_cost0.677 ↓65$0.000000 · 5,271 tokp50 91,096 · p95 182,205 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b
served on mac
triage_cost0.677 ↓65$0.000000 · 5,271 tokp50 91,096 · p95 182,205 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f
served on mac
triage_cost0.677 ↓65$0.000000 · 5,271 tokp50 91,096 · p95 182,205 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v2
served on x3
triage_cost0.677 ↓65$0.000000 · 6,288 tokp50 7,933 · p95 9,459 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd
served on x3
triage_cost0.677 ↓65$0.000000 · 6,288 tokp50 7,933 · p95 9,459 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c
served on x3
triage_cost0.677 ↓65$0.000000 · 6,288 tokp50 7,933 · p95 9,459 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v1
served on dgx
triage_cost0.692 ↓65$0.000000 · 3,194 tokp50 13,206 · p95 24,077 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.692 ↓65$0.000000 · 3,194 tokp50 13,206 · p95 24,077 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b
served on dgx
triage_cost0.692 ↓65$0.000000 · 3,194 tokp50 13,206 · p95 24,077 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.692 ↓65$0.000000 · 3,194 tokp50 13,206 · p95 24,077 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v2
served on mac
triage_cost0.692 ↓65$0.000000 · 6,265 tokp50 11,237 · p95 14,132 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd
served on mac
triage_cost0.692 ↓65$0.000000 · 6,265 tokp50 11,237 · p95 14,132 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c
served on mac
triage_cost0.692 ↓65$0.000000 · 6,265 tokp50 11,237 · p95 14,132 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v1
served on x3
triage_cost0.692 ↓65$0.000000 · 5,386 tokp50 60,463 · p95 159,924 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c
served on x3
triage_cost0.692 ↓65$0.000000 · 5,386 tokp50 60,463 · p95 159,924 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b
served on x3
triage_cost0.692 ↓65$0.000000 · 5,386 tokp50 60,463 · p95 159,924 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f
served on x3
triage_cost0.692 ↓65$0.000000 · 5,386 tokp50 60,463 · p95 159,924 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen3_30b_a3b_v1
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.000700 · 6,111 tokp50 16,982 · p95 35,517 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.000700 · 6,111 tokp50 16,982 · p95 35,517 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.000700 · 6,111 tokp50 16,982 · p95 35,517 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.000700 · 6,111 tokp50 16,982 · p95 35,517 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.000700 · 6,111 tokp50 16,982 · p95 35,517 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v2
served on mac
triage_cost0.708 ↓65$0.000000 · 3,270 tokp50 5,856 · p95 8,006 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd
served on mac
triage_cost0.708 ↓65$0.000000 · 3,270 tokp50 5,856 · p95 8,006 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c
served on mac
triage_cost0.708 ↓65$0.000000 · 3,270 tokp50 5,856 · p95 8,006 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v2
served on x3
triage_cost0.708 ↓65$0.000000 · 4,219 tokp50 14,550 · p95 21,031 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd
served on x3
triage_cost0.708 ↓65$0.000000 · 4,219 tokp50 14,550 · p95 21,031 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c
served on x3
triage_cost0.708 ↓65$0.000000 · 4,219 tokp50 14,550 · p95 21,031 ms1gold/orrery_triage_v1_labelled goldinspect
triage_sonnet_55_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.022659 · 7,149 tokp50 12,111 · p95 20,225 ms1gold/orrery_triage_v1_labelled goldinspect
triage_sonnet_55_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.022659 · 7,149 tokp50 12,111 · p95 20,225 ms1gold/orrery_triage_v1_labelled goldinspect
triage_sonnet_55_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.022659 · 7,149 tokp50 12,111 · p95 20,225 ms1gold/orrery_triage_v1_labelled goldinspect
triage_sonnet_55_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.708 ↓65$0.022659 · 7,149 tokp50 12,111 · p95 20,225 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.723 ↓65$0.000903 · 4,276 tokp50 9,818 · p95 19,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.723 ↓65$0.000903 · 4,276 tokp50 9,818 · p95 19,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.723 ↓65$0.000903 · 4,276 tokp50 9,818 · p95 19,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.723 ↓65$0.000897 · 4,242 tokp50 6,789 · p95 32,394 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.723 ↓65$0.000897 · 4,242 tokp50 6,789 · p95 32,394 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.723 ↓65$0.000897 · 4,242 tokp50 6,789 · p95 32,394 ms1gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.723 ↓65$0.012041 · 4,605 tokp50 22,561 · p95 34,524 ms1gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.723 ↓65$0.012041 · 4,605 tokp50 22,561 · p95 34,524 ms1gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.723 ↓65$0.012041 · 4,605 tokp50 22,561 · p95 34,524 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v2
served on dgx
triage_cost0.723 ↓65$0.000000 · 4,333 tokp50 12,714 · p95 18,314 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd
served on dgx
triage_cost0.723 ↓65$0.000000 · 4,333 tokp50 12,714 · p95 18,314 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c
served on dgx
triage_cost0.723 ↓65$0.000000 · 4,333 tokp50 12,714 · p95 18,314 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v2
served on dgx
triage_cost0.723 ↓65$0.000000 · 2,977 tokp50 4,298 · p95 6,526 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd
served on dgx
triage_cost0.723 ↓65$0.000000 · 2,977 tokp50 4,298 · p95 6,526 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c
served on dgx
triage_cost0.723 ↓65$0.000000 · 2,977 tokp50 4,298 · p95 6,526 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen3_30b_a3b_v1
served on x3
triage_cost0.723 ↓65$0.000000 · 6,231 tokp50 6,349 · p95 10,328 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd
served on x3
triage_cost0.723 ↓65$0.000000 · 6,231 tokp50 6,349 · p95 10,328 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c
served on x3
triage_cost0.723 ↓65$0.000000 · 6,231 tokp50 6,349 · p95 10,328 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b
served on x3
triage_cost0.723 ↓65$0.000000 · 6,231 tokp50 6,349 · p95 10,328 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f
served on x3
triage_cost0.723 ↓65$0.000000 · 6,231 tokp50 6,349 · p95 10,328 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.754 ↓65$0.000862 · 5,180 tokp50 6,905 · p95 27,865 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.754 ↓65$0.000862 · 5,180 tokp50 6,905 · p95 27,865 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.754 ↓65$0.000862 · 5,180 tokp50 6,905 · p95 27,865 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v1
served on mac
triage_cost0.754 ↓65$0.000000 · 6,321 tokp50 21,874 · p95 33,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c
served on mac
triage_cost0.754 ↓65$0.000000 · 6,321 tokp50 21,874 · p95 33,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b
served on mac
triage_cost0.754 ↓65$0.000000 · 6,321 tokp50 21,874 · p95 33,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f
served on mac
triage_cost0.754 ↓65$0.000000 · 6,321 tokp50 21,874 · p95 33,411 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_phi4_14b_v1
served on mac
triage_cost0.754 ↓65$0.000000 · 5,888 tokp50 22,493 · p95 27,539 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd
served on mac
triage_cost0.754 ↓65$0.000000 · 5,888 tokp50 22,493 · p95 27,539 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c
served on mac
triage_cost0.754 ↓65$0.000000 · 5,888 tokp50 22,493 · p95 27,539 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_phi4_14b_v1 · rescored 77adc97b
served on mac
triage_cost0.754 ↓65$0.000000 · 5,888 tokp50 22,493 · p95 27,539 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f
served on mac
triage_cost0.754 ↓65$0.000000 · 5,888 tokp50 22,493 · p95 27,539 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.764 ↓65$0.005104 · 4,916 tokp50 21,498 · p95 55,678 ms3gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.764 ↓65$0.005104 · 4,916 tokp50 21,498 · p95 55,678 ms3gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.764 ↓65$0.005104 · 4,916 tokp50 21,498 · p95 55,678 ms3gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3
run on Linux/x86_64, 32 CPUs
triage_cost0.764 ↓65$0.005104 · 4,916 tokp50 21,498 · p95 55,678 ms3gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_phi4_14b_v1
served on x3
triage_cost0.769 ↓65$0.000000 · 5,949 tokp50 16,420 · p95 21,153 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd
served on x3
triage_cost0.769 ↓65$0.000000 · 5,949 tokp50 16,420 · p95 21,153 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c
served on x3
triage_cost0.769 ↓65$0.000000 · 5,949 tokp50 16,420 · p95 21,153 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_phi4_14b_v1 · rescored 77adc97b
served on x3
triage_cost0.769 ↓65$0.000000 · 5,949 tokp50 16,420 · p95 21,153 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f
served on x3
triage_cost0.769 ↓65$0.000000 · 5,949 tokp50 16,420 · p95 21,153 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.785 ↓65$0.004992 · 4,790 tokp50 19,965 · p95 47,741 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored 5f2c057c baseline
run on Linux/x86_64, 32 CPUs
triage_cost0.785 ↓65$0.004992 · 4,790 tokp50 19,965 · p95 47,741 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.785 ↓65$0.004992 · 4,790 tokp50 19,965 · p95 47,741 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.785 ↓65$0.004992 · 4,790 tokp50 19,965 · p95 47,741 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v2
served on mac
triage_cost0.800 ↓65$0.000000 · 4,345 tokp50 24,585 · p95 36,311 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd
served on mac
triage_cost0.800 ↓65$0.000000 · 4,345 tokp50 24,585 · p95 36,311 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c
served on mac
triage_cost0.800 ↓65$0.000000 · 4,345 tokp50 24,585 · p95 36,311 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.800 ↓65$0.006867 · 6,704 tokp50 15,459 · p95 30,084 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.800 ↓65$0.006867 · 6,704 tokp50 15,459 · p95 30,084 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.800 ↓65$0.006867 · 6,704 tokp50 15,459 · p95 30,084 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.800 ↓65$0.006948 · 6,760 tokp50 19,158 · p95 35,678 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.800 ↓65$0.006948 · 6,760 tokp50 19,158 · p95 35,678 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.800 ↓65$0.006948 · 6,760 tokp50 19,158 · p95 35,678 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v2
served on x3
triage_cost0.815 ↓65$0.000000 · 3,225 tokp50 6,092 · p95 8,461 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd
served on x3
triage_cost0.815 ↓65$0.000000 · 3,225 tokp50 6,092 · p95 8,461 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c
served on x3
triage_cost0.815 ↓65$0.000000 · 3,225 tokp50 6,092 · p95 8,461 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_ministral_3_14b_v1
served on x3
triage_cost0.815 ↓65$0.000000 · 6,617 tokp50 22,275 · p95 31,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd
served on x3
triage_cost0.815 ↓65$0.000000 · 6,617 tokp50 22,275 · p95 31,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c
served on x3
triage_cost0.815 ↓65$0.000000 · 6,617 tokp50 22,275 · p95 31,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b
served on x3
triage_cost0.815 ↓65$0.000000 · 6,617 tokp50 22,275 · p95 31,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f
served on x3
triage_cost0.815 ↓65$0.000000 · 6,617 tokp50 22,275 · p95 31,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_ministral_3_14b_v1
served on dgx
triage_cost0.831 ↓65$0.000000 · 6,489 tokp50 19,704 · p95 29,658 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd
served on dgx
triage_cost0.831 ↓65$0.000000 · 6,489 tokp50 19,704 · p95 29,658 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.831 ↓65$0.000000 · 6,489 tokp50 19,704 · p95 29,658 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b
served on dgx
triage_cost0.831 ↓65$0.000000 · 6,489 tokp50 19,704 · p95 29,658 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.831 ↓65$0.000000 · 6,489 tokp50 19,704 · p95 29,658 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v1
served on dgx
triage_cost0.831 ↓65$0.000000 · 4,536 tokp50 33,735 · p95 60,085 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.831 ↓65$0.000000 · 4,536 tokp50 33,735 · p95 60,085 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b
served on dgx
triage_cost0.831 ↓65$0.000000 · 4,536 tokp50 33,735 · p95 60,085 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.831 ↓65$0.000000 · 4,536 tokp50 33,735 · p95 60,085 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_ministral_3_14b_v1
served on mac
triage_cost0.831 ↓65$0.000000 · 6,497 tokp50 27,548 · p95 39,331 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f
served on mac
triage_cost0.831 ↓65$0.000000 · 6,497 tokp50 27,548 · p95 39,331 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen3_30b_a3b_v1
served on mac
triage_cost0.831 ↓65$0.000000 · 6,279 tokp50 8,898 · p95 13,952 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd
served on mac
triage_cost0.831 ↓65$0.000000 · 6,279 tokp50 8,898 · p95 13,952 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c
served on mac
triage_cost0.831 ↓65$0.000000 · 6,279 tokp50 8,898 · p95 13,952 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b
served on mac
triage_cost0.831 ↓65$0.000000 · 6,279 tokp50 8,898 · p95 13,952 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f
served on mac
triage_cost0.831 ↓65$0.000000 · 6,279 tokp50 8,898 · p95 13,952 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_dismiss_v1
run on Linux/x86_64, 32 CPUs
triage_cost0.862 ↓651,100 tokp50 0.092 · p95 0.17 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_dismiss_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.862 ↓651,100 tokp50 0.092 · p95 0.17 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_dismiss_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.862 ↓651,100 tokp50 0.092 · p95 0.17 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_dismiss_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.862 ↓651,100 tokp50 0.092 · p95 0.17 ms1gold/orrery_triage_v1_labelled goldinspect
triage_stub_dismiss_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.862 ↓651,100 tokp50 0.092 · p95 0.17 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd
run on Linux/x86_64, 32 CPUs
triage_cost0.877 ↓65$0.011539 · 6,292 tokp50 6,080 · p95 8,340 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost0.877 ↓65$0.011539 · 6,292 tokp50 6,080 · p95 8,340 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost0.877 ↓65$0.011539 · 6,292 tokp50 6,080 · p95 8,340 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost0.877 ↓65$0.011539 · 6,292 tokp50 6,080 · p95 8,340 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.877 ↓65$0.001654 · 4,247 tokp50 6,774 · p95 9,273 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.877 ↓65$0.001654 · 4,247 tokp50 6,774 · p95 9,273 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.877 ↓65$0.001654 · 4,247 tokp50 6,774 · p95 9,273 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen3_30b_a3b_v1
served on dgx
triage_cost0.892 ↓65$0.000000 · 6,384 tokp50 5,362 · p95 8,443 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd
served on dgx
triage_cost0.892 ↓65$0.000000 · 6,384 tokp50 5,362 · p95 8,443 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c
served on dgx
triage_cost0.892 ↓65$0.000000 · 6,384 tokp50 5,362 · p95 8,443 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b
served on dgx
triage_cost0.892 ↓65$0.000000 · 6,384 tokp50 5,362 · p95 8,443 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f
served on dgx
triage_cost0.892 ↓65$0.000000 · 6,384 tokp50 5,362 · p95 8,443 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v2
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.908 ↓65$0.000355 · 4,756 tokp50 2,897 · p95 6,148 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v2 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.908 ↓65$0.000355 · 4,756 tokp50 2,897 · p95 6,148 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v2 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost0.908 ↓65$0.000355 · 4,756 tokp50 2,897 · p95 6,148 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd
served on mac
triage_cost0.908 ↓65$0.000000 · 6,497 tokp50 27,548 · p95 39,331 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c
served on mac
triage_cost0.908 ↓65$0.000000 · 6,497 tokp50 27,548 · p95 39,331 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b
served on mac
triage_cost0.908 ↓65$0.000000 · 6,497 tokp50 27,548 · p95 39,331 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v1
served on x3
triage_cost0.908 ↓65$0.000000 · 4,388 tokp50 43,283 · p95 76,811 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c
served on x3
triage_cost0.908 ↓65$0.000000 · 4,388 tokp50 43,283 · p95 76,811 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b
served on x3
triage_cost0.908 ↓65$0.000000 · 4,388 tokp50 43,283 · p95 76,811 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f
served on x3
triage_cost0.908 ↓65$0.000000 · 4,388 tokp50 43,283 · p95 76,811 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_mistral_medium_35_128b_v1
served on dgx
triage_cost1.000 ↓65$0.000000 · 5,941 tokp50 112,303 · p95 167,929 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd
served on dgx
triage_cost1.000 ↓65$0.000000 · 5,941 tokp50 112,303 · p95 167,929 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c
served on dgx
triage_cost1.000 ↓65$0.000000 · 5,941 tokp50 112,303 · p95 167,929 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b
served on dgx
triage_cost1.000 ↓65$0.000000 · 5,941 tokp50 112,303 · p95 167,929 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f
served on dgx
triage_cost1.000 ↓65$0.000000 · 5,941 tokp50 112,303 · p95 167,929 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c
served on dgx
triage_cost1.015 ↓65$0.000000 · 3,538 tokp50 12,488 · p95 21,540 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b
served on dgx
triage_cost1.015 ↓65$0.000000 · 3,538 tokp50 12,488 · p95 21,540 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v1
served on mac
triage_cost1.046 ↓65$0.000000 · 4,483 tokp50 62,523 · p95 121,029 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c
served on mac
triage_cost1.046 ↓65$0.000000 · 4,483 tokp50 62,523 · p95 121,029 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b
served on mac
triage_cost1.046 ↓65$0.000000 · 4,483 tokp50 62,523 · p95 121,029 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f
served on mac
triage_cost1.046 ↓65$0.000000 · 4,483 tokp50 62,523 · p95 121,029 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c
run on Linux/x86_64, 32 CPUs
triage_cost1.077 ↓65$0.003350 · 5,474 tokp50 15,600 · p95 33,509 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v1 · rescored 77adc97b
run on Linux/x86_64, 32 CPUs
triage_cost1.077 ↓65$0.003350 · 5,474 tokp50 15,600 · p95 33,509 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f
run on Linux/x86_64, 32 CPUs
triage_cost1.077 ↓65$0.003350 · 5,474 tokp50 15,600 · p95 33,509 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c
served on mac
triage_cost1.092 ↓65$0.000000 · 3,429 tokp50 17,421 · p95 30,731 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b
served on mac
triage_cost1.092 ↓65$0.000000 · 3,429 tokp50 17,421 · p95 30,731 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c
served on x3
triage_cost1.092 ↓65$0.000000 · 3,619 tokp50 16,207 · p95 29,798 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b
served on x3
triage_cost1.092 ↓65$0.000000 · 3,619 tokp50 16,207 · p95 29,798 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v1
served on dgx
triage_cost1.138 ↓65$0.000000 · 5,318 tokp50 35,972 · p95 57,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c
served on dgx
triage_cost1.138 ↓65$0.000000 · 5,318 tokp50 35,972 · p95 57,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b
served on dgx
triage_cost1.138 ↓65$0.000000 · 5,318 tokp50 35,972 · p95 57,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f
served on dgx
triage_cost1.138 ↓65$0.000000 · 5,318 tokp50 35,972 · p95 57,784 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v2
served on dgx
triage_cost1.138 ↓65$0.000000 · 4,585 tokp50 15,771 · p95 24,424 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd
served on dgx
triage_cost1.138 ↓65$0.000000 · 4,585 tokp50 15,771 · p95 24,424 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c
served on dgx
triage_cost1.138 ↓65$0.000000 · 4,585 tokp50 15,771 · p95 24,424 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v2
served on mac
triage_cost1.138 ↓65$0.000000 · 4,569 tokp50 32,196 · p95 46,026 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd
served on mac
triage_cost1.138 ↓65$0.000000 · 4,569 tokp50 32,196 · p95 46,026 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c
served on mac
triage_cost1.138 ↓65$0.000000 · 4,569 tokp50 32,196 · p95 46,026 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_noreason_v1
served on homelab LiteLLM (the fleet's gateway)
triage_cost1.154 ↓65$0.004862 · 5,590 tokp50 7,710 · p95 14,703 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd
served on homelab LiteLLM (the fleet's gateway)
triage_cost1.154 ↓65$0.004862 · 5,590 tokp50 7,710 · p95 14,703 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_noreason_v1 · rescored 5f2c057c
served on homelab LiteLLM (the fleet's gateway)
triage_cost1.154 ↓65$0.004862 · 5,590 tokp50 7,710 · p95 14,703 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama4_scout_v1
served on dgx
triage_cost1.262 ↓65$0.000000 · 5,629 tokp50 13,116 · p95 17,645 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd
served on dgx
triage_cost1.262 ↓65$0.000000 · 5,629 tokp50 13,116 · p95 17,645 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c
served on dgx
triage_cost1.262 ↓65$0.000000 · 5,629 tokp50 13,116 · p95 17,645 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b
served on dgx
triage_cost1.262 ↓65$0.000000 · 5,629 tokp50 13,116 · p95 17,645 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f
served on dgx
triage_cost1.262 ↓65$0.000000 · 5,629 tokp50 13,116 · p95 17,645 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v2
served on x3
triage_cost1.292 ↓65$0.000000 · 4,523 tokp50 19,431 · p95 31,568 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd
served on x3
triage_cost1.292 ↓65$0.000000 · 4,523 tokp50 19,431 · p95 31,568 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c
served on x3
triage_cost1.292 ↓65$0.000000 · 4,523 tokp50 19,431 · p95 31,568 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v2
served on dgx
triage_cost1.354 ↓65$0.000000 · 4,180 tokp50 15,011 · p95 19,803 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd
served on dgx
triage_cost1.354 ↓65$0.000000 · 4,180 tokp50 15,011 · p95 19,803 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c
served on dgx
triage_cost1.354 ↓65$0.000000 · 4,180 tokp50 15,011 · p95 19,803 ms1gold/orrery_triage_v1_labelled goldinspect
triage_local_x3_llama31_8b_v1
served on x3
false_dismiss0.000 ↓34$0.000000 · 5,490 tokp50 5,389 · p95 8,370 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_35_flash_lite_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.088 ↓34$0.001368 · 3,125 tokp50 3,353 · p95 5,695 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1 · repeats ×3
run on Linux/x86_64, 32 CPUs
false_dismiss0.088 ↓34$0.010918 · 4,160 tokp50 11,696 · p95 18,138 ms3gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.088 ↓34$0.000715 · 4,764 tokp50 11,518 · p95 21,428 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gemini_4_argon_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.118 ↓34$0.013617 · 4,751 tokp50 23,508 · p95 61,827 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_61_sol_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.118 ↓34$0.010847 · 4,156 tokp50 12,311 · p95 19,302 ms1gold/orrery_triage_v1_labelled goldinspect
triage_gpt_6_luna_v1 · repeats ×3
run on Linux/x86_64, 32 CPUs
false_dismiss0.118 ↓34$0.000684 · 4,621 tokp50 10,962 · p95 18,893 ms3gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.118 ↓34$0.000998 · 5,684 tokp50 6,867 · p95 13,887 ms1gold/orrery_triage_v1_labelled goldinspect
triage_haiku_55_v1 · repeats ×3
run on Linux/x86_64, 32 CPUs
false_dismiss0.118 ↓34$0.001024 · 5,815 tokp50 7,273 · p95 12,605 ms3gold/orrery_triage_v1_labelled goldinspect
triage_kimi_k31_pro_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.118 ↓34$0.019130 · 5,576 tokp50 43,158 · p95 79,162 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gpt_oss_120b_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.147 ↓34$0.000849 · 3,854 tokp50 16,361 · p95 31,807 ms1gold/orrery_triage_v1_labelled goldinspect
triage_sonnet_55_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.147 ↓34$0.022659 · 7,149 tokp50 12,111 · p95 20,225 ms1gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 · repeats ×3
run on Linux/x86_64, 32 CPUs
false_dismiss0.167 ↓34$0.005104 · 4,916 tokp50 21,498 · p95 55,678 ms3gold/orrery_triage_v1_labelled goldinspect
triage_baseline_pro_v1 baseline
run on Linux/x86_64, 32 CPUs
false_dismiss0.176 ↓34$0.004992 · 4,790 tokp50 19,965 · p95 47,741 ms1gold/orrery_triage_v1_labelled goldinspect
triage_deepseek_41flash_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.176 ↓34$0.000862 · 5,180 tokp50 6,905 · p95 27,865 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_gemma4_31b_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.176 ↓34$0.000903 · 4,276 tokp50 9,818 · p95 19,215 ms1gold/orrery_triage_v1_labelled goldinspect
triage_mistral_large_4_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.176 ↓34$0.006867 · 6,704 tokp50 15,459 · p95 30,084 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_mistral_medium_35_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.235 ↓34$0.011539 · 6,292 tokp50 6,080 · p95 8,340 ms1gold/orrery_triage_v1_labelled goldinspect
triage_hosted_qwen35_122b_v1
run on Linux/x86_64, 32 CPUs
false_dismiss0.294 ↓34$0.003350 · 5,474 tokp50 15,600 · p95 33,509 ms1gold/orrery_triage_v1_labelled goldinspect

02what's inside

65 items, json, 62–440 words (median 149, middle half 128–208).

0102062–99 words: 12100–137 words: 15138–175 words: 15176–213 words: 14214–251 words: 3366–403 words: 4404–441 words: 262–99100–137138–175176–213214–251252–289290–327328–365366–403404–441

the gold

gold/orrery_triage_v1_labelled: 4 labels

labelitemsshare
dismiss2335%
{"accept": ["escalate", "file"]}2335%
file1117%
cleanup812%
65 shown
itemwordsgoldopen
orrery-ORRER68{"accept": ["escalate", "file"]}inspect
orrery-ORRER88{"accept": ["escalate", "file"]}inspect
orrery-ORRER72{"accept": ["escalate", "file"]}inspect
orrery-ORRER68{"accept": ["escalate", "file"]}inspect
orrery-ORRER72{"accept": ["escalate", "file"]}inspect
orrery-ORRER76{"accept": ["escalate", "file"]}inspect
orrery-ORRER62{"accept": ["escalate", "file"]}inspect
orrery-ORRER84{"accept": ["escalate", "file"]}inspect
orrery-ORRER72{"accept": ["escalate", "file"]}inspect
orrery-ORRER68{"accept": ["escalate", "file"]}inspect
orrery-ORRER72{"accept": ["escalate", "file"]}inspect
orrery-ORRER128{"accept": ["escalate", "file"]}inspect
orrery-ORRER165{"accept": ["escalate", "file"]}inspect
orrery-ORRER128{"accept": ["escalate", "file"]}inspect
orrery-ORRER132{"accept": ["escalate", "file"]}inspect
orrery-ORRER128{"accept": ["escalate", "file"]}inspect
orrery-ORRER164dismissinspect
orrery-ORRER168{"accept": ["escalate", "file"]}inspect
orrery-ORRER118{"accept": ["escalate", "file"]}inspect
orrery-ORRER140dismissinspect
orrery-ORRER140dismissinspect
orrery-ORRER152dismissinspect
orrery-ORRER122dismissinspect
orrery-ORRER170{"accept": ["escalate", "file"]}inspect
orrery-ORRER118{"accept": ["escalate", "file"]}inspect
orrery-ORRER172{"accept": ["escalate", "file"]}inspect
orrery-ORRER132dismissinspect
orrery-ORRER132dismissinspect
orrery-ORRER132dismissinspect
orrery-ORRER132dismissinspect
orrery-ORRER149dismissinspect
orrery-ORRER149dismissinspect
orrery-ORRER125{"accept": ["escalate", "file"]}inspect
orrery-ORRER145dismissinspect
orrery-ORRER181{"accept": ["escalate", "file"]}inspect
orrery-ORRER223dismissinspect
orrery-ORRER223fileinspect
orrery-ORRER220dismissinspect
orrery-ORRER195dismissinspect
orrery-ORRER195dismissinspect
orrery-ORRER149cleanupinspect
orrery-ORRER440dismissinspect
orrery-ORRER204fileinspect
orrery-ORRER161cleanupinspect
orrery-ORRER397cleanupinspect
orrery-ORRER167cleanupinspect
orrery-ORRER211fileinspect
orrery-ORRER211fileinspect
orrery-ORRER377dismissinspect
orrery-ORRER211fileinspect
orrery-ORRER211fileinspect
orrery-ORRER212fileinspect
orrery-ORRER211fileinspect
orrery-ORRER395fileinspect
orrery-ORRER211fileinspect
orrery-ORRER405dismissinspect
orrery-ORRER99cleanupinspect
orrery-ORRER132cleanupinspect
orrery-ORRER129cleanupinspect
orrery-ORRER137cleanupinspect
orrery-ORRER208dismissinspect
orrery-ORRER208dismissinspect
orrery-ORRER208dismissinspect
orrery-ORRER401dismissinspect
orrery-ORRER170fileinspect

Each row is the profile evh dataset-items --dataset-id orrery_triage_v1_labelled prints; inspect opens the item in the inspector with the dataset's largest group of comparable arms, and its text where the inspector runs on a machine that has it.

04labels

setmade oncoverageguidelinemadepromoted
person gold_orrery_triage_smokeorrery_triage_smoke2 of 65noneperson · owner 2026-11-01gold/orrery_triage_smoke gold
person gold_orrery_triage_v1_labelledthis one65 of 65noneperson · owner 2026-11-01gold/orrery_triage_v1_labelled gold
person silver_orrery_triage_v1_labelled_handthis one3 of 65noneperson · fixture labels 2026-11-01silver/orrery_triage_v1_labelled_hand silver
rule silver_orrery_triage_v1_labelled_outcomesthis one34 of 65outcomes.yamlrule · orrery_2026_10 2026-11-02silver/orrery_triage_v1_labelled_outcomes silver

05item fields

Declared in the field set triage. Values are summarised only for fields marked public; the inspector shows each item's, filters a comparison by them, and breaks it down by a choice or yes/no field.

fieldtypefilleddescriptionvalues
Evidence
evidence_level
choice65 of 65exact: the probe answers the model saw; the rest rebuilt for the decision's time, or unavailable.exact 30 · no-logs 23 · ledger-only 11 · no-traces 1
Decided by
decision_kind
choice65 of 65model: the model decided; triager_failure: it never did (the 2026-08 authentication outage).model 61 · triager_failure 4
Production decided
fleet_disposition
choice65 of 65What the fleet decided in production.file 29 · escalate 24 · cleanup 8 · dismiss 4
Production model
fleet_model
text65 of 65The gateway alias production used.1 distinct
Stage
stage
text65 of 65The fleet's rollout stage for the decision (propose, shadow; empty for a test signal).1 distinct
Fingerprint
fingerprint
text65 of 65The signal's fingerprint; decisions on one fingerprint are related.65 distinct
Decided on
decision_date
date65 of 65The day the fleet decided.2026-07-30 – 2026-10-29
Owner-labelled
labelled
choice65 of 65yes: the owner's label exists for this decision (verdicts.jsonl, source owner-*).yes 65