eval-harness

metric

triage_cost

better
lower
kind
quality: measured against a reference or by a judge
decides
Triage decisions on Orrery's error signals
weighed by
no goal
recorded on
orrery_triage_v1_labelled

01what it measures, and how

One decision's cost: 5 for a missed real problem, 1 for a false alarm, 0 when right; lower is better.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"triage_cost": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0.4triage_local_x3_gpt_oss_20b_v1: 0.4triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0.4triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0.4triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0.4triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0.4triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0.4triage_local_dgx_llama31_8b_v1: 0.415triage_local_mac_llama31_8b_v1: 0.415triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0.415triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0.415triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0.415triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0.415triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0.415triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0.415triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0.415triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0.415triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0.415triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0.415triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0.415triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0.415triage_local_dgx_gpt_oss_20b_v1: 0.477triage_stub_escalate_v1: 0.477triage_stub_escalate_v1 · rescored 3d3a6afd: 0.477triage_stub_escalate_v1 · rescored 5f2c057c: 0.477triage_stub_escalate_v1 · rescored 77adc97b: 0.477triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0.477triage_stub_escalate_v1 · rescored cd35dc1f: 0.477triage_local_dgx_nemotron_35_bolt_e4b_v1: 0.523triage_local_x3_nemotron_35_lightning_30b_v1: 0.523triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.523triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.523triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.523triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.523triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.523triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.523triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.538triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.538triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.538triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.538triage_gpt_6_luna_v2: 0.554triage_local_mac_gpt_oss_20b_v1: 0.554triage_local_x3_nemotron_35_bolt_e4b_v2: 0.554triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.554triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.554triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.554triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.554triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0.554triage_local_dgx_gemma4_12b_noreason_v1: 0.585triage_local_mac_gemma4_12b_noreason_v1: 0.585triage_local_mac_nemotron_35_bolt_e4b_v2: 0.585triage_haiku_55_v1 · rescored 3d3a6afd: 0.585triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.585triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.585triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.585triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.585triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.585triage_haiku_55_v1 · rescored 5f2c057c: 0.585triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.585triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.585triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.585triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.585triage_gpt_6_luna_v1 · rescored 77adc97b: 0.585triage_haiku_55_v1 · rescored 77adc97b: 0.585triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.585triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.585triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.585triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.585triage_haiku_55_v1 · rescored cd35dc1f: 0.585triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.585triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.585triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.585triage_local_x3_nemotron_35_bolt_e4b_v1: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.6triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0.6triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.61triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.61triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.61triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.61triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.626triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.626triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.626triage_hosted_gpt_oss_120b_v2: 0.631triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.631triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.631triage_local_dgx_nemotron_35_bolt_e4b_v2: 0.646triage_local_dgx_nemotron_35_lightning_30b_v1: 0.646triage_local_mac_nemotron_35_lightning_30b_v1: 0.646triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.646triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.646triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.646triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.646triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.646triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.646triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.646triage_gemini_4_argon_v1 · rescored 77adc97b: 0.646triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.646triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.646triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.646triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.646triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.646triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.646triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.646triage_local_dgx_gpt_oss_120b_v2: 0.662triage_local_dgx_phi4_14b_v1: 0.662triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.662triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.662triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0.662triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.662triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0.662triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0.662triage_gpt_61_sol_v1 · rescored 77adc97b: 0.662triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0.662triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.662triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0.662triage_local_dgx_gemma4_31b_v1: 0.677triage_local_dgx_nemotron_35_lightning_30b_v2: 0.677triage_local_dgx_nemotron_35_super_120b_v1: 0.677triage_local_dgx_nemotron_35_super_120b_v2: 0.677triage_local_mac_gemma4_31b_v1: 0.677triage_local_x3_nemotron_35_lightning_30b_v2: 0.677triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.677triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0.677triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.677triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.677triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0.677triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.677triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0.677triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0.677triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0.677triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.677triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.677triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0.677triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0.677triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0.677triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.677triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0.677triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0.677triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0.677triage_local_dgx_gpt_oss_120b_v1: 0.692triage_local_mac_nemotron_35_lightning_30b_v2: 0.692triage_local_x3_gemma4_31b_v1: 0.692triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.692triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0.692triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.692triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0.692triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0.692triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0.692triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0.692triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0.692triage_hosted_qwen3_30b_a3b_v1: 0.708triage_local_mac_gpt_oss_20b_v2: 0.708triage_local_x3_gemma4_31b_v2: 0.708triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.708triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.708triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0.708triage_sonnet_55_v1 · rescored 3d3a6afd: 0.708triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.708triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0.708triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0.708triage_sonnet_55_v1 · rescored 5f2c057c: 0.708triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.708triage_sonnet_55_v1 · rescored 77adc97b: 0.708triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.708triage_sonnet_55_v1 · rescored cd35dc1f: 0.708triage_hosted_gemma4_31b_v2: 0.723triage_kimi_k31_pro_v2: 0.723triage_local_dgx_gemma4_31b_v2: 0.723triage_local_dgx_gpt_oss_20b_v2: 0.723triage_local_x3_qwen3_30b_a3b_v1: 0.723triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.723triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.723triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0.723triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.723triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.723triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.723triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.723triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.723triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0.723triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0.723triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.723triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.723triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.723triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.723triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.723triage_local_mac_nemotron_35_bolt_e4b_v1: 0.754triage_local_mac_phi4_14b_v1: 0.754triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0.754triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.754triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.754triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0.754triage_deepseek_41flash_v1 · rescored 77adc97b: 0.754triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.754triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0.754triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.754triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.754triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0.754triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.764triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.764triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.764triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.764triage_local_x3_phi4_14b_v1: 0.769triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0.769triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0.769triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0.769triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0.769triage_baseline_pro_v1 · rescored 3d3a6afd: 0.785triage_baseline_pro_v1 · rescored 5f2c057c: 0.785triage_baseline_pro_v1 · rescored 77adc97b: 0.785triage_baseline_pro_v1 · rescored cd35dc1f: 0.785triage_local_mac_gemma4_31b_v2: 0.8triage_mistral_large_4_v2: 0.8triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0.8triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.8triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0.8triage_mistral_large_4_v1 · rescored 5f2c057c: 0.8triage_mistral_large_4_v2 · rescored 5f2c057c: 0.8triage_mistral_large_4_v1 · rescored 77adc97b: 0.8triage_mistral_large_4_v1 · rescored cd35dc1f: 0.8triage_local_x3_gpt_oss_20b_v2: 0.815triage_local_x3_ministral_3_14b_v1: 0.815triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.815triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0.815triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0.815triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0.815triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0.815triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0.815triage_local_dgx_ministral_3_14b_v1: 0.831triage_local_dgx_qwen38_27b_v1: 0.831triage_local_mac_ministral_3_14b_v1: 0.831triage_local_mac_qwen3_30b_a3b_v1: 0.831triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0.831triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.831triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0.831triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0.831triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.831triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0.831triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0.831triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.831triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0.831triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0.831triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0.831triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.831triage_stub_dismiss_v1: 0.862triage_stub_dismiss_v1 · rescored 3d3a6afd: 0.862triage_stub_dismiss_v1 · rescored 5f2c057c: 0.862triage_stub_dismiss_v1 · rescored 77adc97b: 0.862triage_stub_dismiss_v1 · rescored cd35dc1f: 0.862triage_hosted_qwen35_122b_v2: 0.877triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.877triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.877triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.877triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.877triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.877triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.877triage_local_dgx_qwen3_30b_a3b_v1: 0.892triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.892triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.892triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.892triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.892triage_deepseek_41flash_v2: 0.908triage_local_x3_qwen38_27b_v1: 0.908triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.908triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0.908triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.908triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0.908triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0.908triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0.908triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0.908triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0.908triage_local_dgx_mistral_medium_35_128b_v1: 1triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 1triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 1triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 1triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 1triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 1.015triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 1.015triage_local_mac_qwen38_27b_v1: 1.046triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 1.046triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 1.046triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 1.046triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 1.077triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 1.077triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 1.077triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 1.092triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 1.092triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 1.092triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 1.092triage_local_dgx_qwen35_122b_v1: 1.138triage_local_dgx_qwen38_27b_v2: 1.138triage_local_mac_qwen38_27b_v2: 1.138triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 1.138triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 1.138triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 1.138triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 1.138triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 1.138triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 1.138triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 1.138triage_baseline_pro_noreason_v1: 1.154triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 1.154triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 1.154triage_local_dgx_llama4_scout_v1: 1.262triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 1.262triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 1.262triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 1.262triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 1.262triage_local_x3_qwen38_27b_v2: 1.292triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 1.292triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 1.292triage_local_dgx_qwen35_122b_v2: 1.354triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 1.354triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 1.3540.4 · better1.354

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean triage_costitemsscored against
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v10.40065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v10.40065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd0.40065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c0.40065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b0.40065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f0.40065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f0.40065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v10.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v10.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v10.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v10.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v20.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v10.55465gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v20.55465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f0.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v10.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v10.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v20.58565gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×30.58565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×30.58565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×30.58565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×30.58565gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v10.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats0.60065gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×30.61065gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×30.61065gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×30.61065gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×30.61065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×30.62665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×30.62665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×30.62665gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v20.63165gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd0.63165gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v20.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v10.64665gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v10.64665gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd0.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.64665gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c0.64665gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c0.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.64665gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.64665gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b0.64665gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b0.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.64665gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.64665gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f0.64665gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f0.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.64665gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.64665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v20.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v10.66265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd0.66265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b0.66265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v10.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v20.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v10.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v20.67765gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v10.67765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v20.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd0.67765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.67765gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b0.67765gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b0.67765gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f0.67765gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f0.67765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v10.69265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v20.69265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v10.69265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.69265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c0.69265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.69265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c0.69265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b0.69265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b0.69265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f0.69265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f0.69265gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v10.70865gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v20.70865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v20.70865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.70865gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd0.70865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd0.70865gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd0.70865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c0.70865gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c0.70865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c0.70865gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c0.70865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b0.70865gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b0.70865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f0.70865gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f0.70865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v20.72365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v20.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v20.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v20.72365gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v10.72365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd0.72365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd0.72365gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.72365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b0.72365gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b0.72365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f0.72365gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f0.72365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v10.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v10.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd0.75465gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c0.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c0.75465gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b0.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b0.75465gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f0.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.75465gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f0.75465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×30.76465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×30.76465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×30.76465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×30.76465gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v10.76965gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd0.76965gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c0.76965gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b0.76965gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f0.76965gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd0.78565gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c0.78565gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b0.78565gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f0.78565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v20.80065gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v20.80065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd0.80065gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd0.80065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c0.80065gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c0.80065gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c0.80065gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b0.80065gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f0.80065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v20.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v10.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd0.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd0.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c0.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c0.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b0.81565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f0.81565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v10.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v10.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v10.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v10.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd0.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c0.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c0.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c0.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b0.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b0.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b0.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f0.83165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f0.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f0.83165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f0.83165gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v10.86265gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd0.86265gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c0.86265gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b0.86265gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f0.86265gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v20.87765gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd0.87765gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd0.87765gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c0.87765gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c0.87765gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b0.87765gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f0.87765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v10.89265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.89265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c0.89265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b0.89265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f0.89265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v20.90865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v10.90865gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd0.90865gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd0.90865gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c0.90865gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c0.90865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c0.90865gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b0.90865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b0.90865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f0.90865gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v11.00065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd1.00065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c1.00065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b1.00065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f1.00065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c1.01565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b1.01565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v11.04665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c1.04665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b1.04665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f1.04665gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c1.07765gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b1.07765gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f1.07765gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c1.09265gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c1.09265gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b1.09265gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b1.09265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v11.13865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v21.13865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v21.13865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd1.13865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd1.13865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c1.13865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c1.13865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c1.13865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b1.13865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f1.13865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v11.15465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd1.15465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c1.15465gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v11.26265gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd1.26265gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c1.26265gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b1.26265gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f1.26265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v21.29265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd1.29265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c1.29265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v21.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd1.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c1.35465gold/orrery_triage_v1_labelled gold