metric

`n_probes`

better: higher
kind: quality: measured against a reference or by a judge
decides: no goal
weighed by: no goal
recorded on: orrery_triage_v1_labelled

01what it measures, and how

How many probes the model asked for before it decided.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"n_probes": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean n_probes	items	scored against
triage_local_dgx_nemotron_35_lightning_30b_v2	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c	3.000	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1	2.938	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd	2.938	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c	2.938	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b	2.938	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f	2.938	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1	2.877	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd	2.877	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c	2.877	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b	2.877	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f	2.877	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1	2.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd	2.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c	2.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b	2.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f	2.862	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f	2.831	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f	2.800	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1	2.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd	2.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c	2.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b	2.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f	2.785	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1	2.769	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd	2.769	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c	2.769	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b	2.769	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f	2.769	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1	2.738	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd	2.738	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c	2.738	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b	2.738	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f	2.738	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1	2.723	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd	2.723	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c	2.723	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b	2.723	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f	2.723	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2	2.677	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd	2.677	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c	2.677	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2	2.600	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd	2.600	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c	2.600	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1	2.554	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c	2.554	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b	2.554	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f	2.554	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1	2.538	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2	2.538	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd	2.538	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd	2.538	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c	2.538	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c	2.538	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b	2.538	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f	2.538	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1	2.508	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd	2.508	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c	2.508	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2	2.492	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd	2.492	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c	2.492	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1	2.400	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c	2.400	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b	2.400	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f	2.400	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1	2.385	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c	2.385	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b	2.385	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f	2.385	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1	2.338	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c	2.338	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b	2.338	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f	2.338	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats	2.323	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2	2.308	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd	2.308	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c	2.308	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1	2.292	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd	2.292	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c	2.292	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b	2.292	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f	2.292	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1	2.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c	2.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b	2.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f	2.277	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1	2.246	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c	2.246	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b	2.246	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f	2.246	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2	2.138	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd	2.138	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c	2.138	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1	2.123	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c	2.123	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b	2.123	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f	2.123	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1	2.108	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c	2.108	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b	2.108	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f	2.108	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1	2.062	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c	2.062	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b	2.062	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f	2.062	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×3	2.041	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3	2.041	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3	2.041	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3	2.041	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1	2.031	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c	2.031	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b	2.031	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f	2.031	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2	2.015	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd	2.015	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c	2.015	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2	1.985	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd	1.985	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c	1.985	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2	1.969	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd	1.969	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c	1.969	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f	1.954	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2	1.908	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd	1.908	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c	1.908	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd	1.892	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c	1.892	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b	1.892	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f	1.892	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f	1.892	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1	1.862	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd	1.862	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f	1.862	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2	1.846	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd	1.846	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c	1.846	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1	1.831	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd	1.831	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c	1.831	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b	1.831	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f	1.831	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×3	1.810	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3	1.810	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3	1.810	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3	1.810	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3	1.810	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2	1.800	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd	1.800	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c	1.800	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2	1.754	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1	1.754	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd	1.754	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd	1.754	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c	1.754	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c	1.754	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b	1.754	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f	1.754	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1	1.738	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c	1.738	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b	1.738	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f	1.738	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×3	1.600	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3	1.600	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3	1.600	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3	1.600	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3	1.600	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1	1.569	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c	1.569	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b	1.569	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f	1.569	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1	1.538	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd	1.538	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c	1.538	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b	1.538	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f	1.538	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1	1.523	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c	1.523	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b	1.523	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f	1.523	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1	1.477	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c	1.477	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b	1.477	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f	1.477	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1	1.446	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c	1.446	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b	1.446	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f	1.446	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2	1.415	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd	1.415	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c	1.415	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1	1.338	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c	1.338	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b	1.338	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f	1.338	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×3	1.333	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3	1.333	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3	1.333	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×3	1.333	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3	1.333	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1	1.323	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd	1.323	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c	1.323	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b	1.323	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f	1.323	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1	1.277	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2	1.277	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd	1.277	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd	1.277	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c	1.277	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c	1.277	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b	1.277	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f	1.277	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2	1.262	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd	1.262	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c	1.262	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1	1.200	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd	1.200	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c	1.200	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b	1.200	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f	1.200	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2	1.185	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd	1.185	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c	1.185	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2	1.123	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd	1.123	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c	1.123	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1	1.031	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c	1.031	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b	1.031	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f	1.031	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f	1.000	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f	1.000	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1	0.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c	0.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b	0.785	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f	0.785	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1	0.738	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c	0.738	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b	0.738	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f	0.738	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1	0.723	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c	0.723	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b	0.723	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f	0.723	65	gold/orrery_triage_v1_labelled gold