eval-harness

metric

triage_cost_p3

better
lower
kind
quality: measured against a reference or by a judge
decides
no goal
weighed by
no goal
recorded on
orrery_triage_v1_labelled

01what it measures, and how

triage_cost with a missed real problem priced at 3 false alarms instead of 5.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"triage_cost_p3": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0.369triage_local_x3_gpt_oss_20b_v1: 0.369triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0.369triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0.369triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0.369triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0.369triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0.369triage_local_dgx_gpt_oss_20b_v1: 0.415triage_local_dgx_llama31_8b_v1: 0.415triage_local_mac_llama31_8b_v1: 0.415triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0.415triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0.415triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0.415triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0.415triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0.415triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0.415triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0.415triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0.415triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0.415triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0.415triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0.415triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0.415triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0.415triage_local_dgx_nemotron_35_bolt_e4b_v1: 0.431triage_local_x3_nemotron_35_bolt_e4b_v2: 0.431triage_local_x3_nemotron_35_lightning_30b_v1: 0.431triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.431triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.431triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.431triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.431triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.431triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.431triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.431triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.431triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.446triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.446triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.446triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.446triage_gpt_6_luna_v2: 0.462triage_local_dgx_gemma4_12b_noreason_v1: 0.462triage_local_mac_gemma4_12b_noreason_v1: 0.462triage_local_mac_gpt_oss_20b_v1: 0.462triage_local_mac_nemotron_35_bolt_e4b_v2: 0.462triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.462triage_haiku_55_v1 · rescored 3d3a6afd: 0.462triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.462triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.462triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.462triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.462triage_haiku_55_v1 · rescored 5f2c057c: 0.462triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.462triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.462triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.462triage_haiku_55_v1 · rescored 77adc97b: 0.462triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.462triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.462triage_haiku_55_v1 · rescored cd35dc1f: 0.462triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.462triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.462triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0.462triage_local_x3_nemotron_35_bolt_e4b_v1: 0.477triage_stub_escalate_v1: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0.477triage_stub_escalate_v1 · rescored 3d3a6afd: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.477triage_stub_escalate_v1 · rescored 5f2c057c: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.477triage_stub_escalate_v1 · rescored 77adc97b: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.477triage_stub_escalate_v1 · rescored cd35dc1f: 0.477triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0.477triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.487triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.487triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.487triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.487triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.492triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.492triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.492triage_gpt_6_luna_v1 · rescored 77adc97b: 0.492triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.492triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.492triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.492triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.503triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.503triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.503triage_hosted_gpt_oss_120b_v2: 0.508triage_local_dgx_gpt_oss_120b_v2: 0.508triage_local_x3_qwen3_30b_a3b_v1: 0.508triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.508triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.508triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.508triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.508triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0.508triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.508triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.508triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.508triage_hosted_qwen3_30b_a3b_v1: 0.523triage_local_dgx_gemma4_31b_v1: 0.523triage_local_dgx_nemotron_35_bolt_e4b_v2: 0.523triage_local_dgx_nemotron_35_lightning_30b_v1: 0.523triage_local_dgx_nemotron_35_super_120b_v1: 0.523triage_local_mac_gemma4_31b_v1: 0.523triage_local_mac_nemotron_35_lightning_30b_v1: 0.523triage_local_x3_gemma4_31b_v2: 0.523triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.523triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.523triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.523triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0.523triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.523triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.523triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.523triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.523triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0.523triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.523triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.523triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0.523triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0.523triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.523triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0.523triage_gemini_4_argon_v1 · rescored 77adc97b: 0.523triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.523triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.523triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.523triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0.523triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.523triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0.523triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0.523triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.523triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.523triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.523triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.523triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.523triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0.523triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.523triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0.523triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0.523triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.523triage_hosted_gemma4_31b_v2: 0.538triage_local_dgx_gemma4_31b_v2: 0.538triage_local_dgx_gpt_oss_120b_v1: 0.538triage_local_dgx_phi4_14b_v1: 0.538triage_local_x3_gemma4_31b_v1: 0.538triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.538triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.538triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0.538triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0.538triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.538triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.538triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.538triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0.538triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0.538triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0.538triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0.538triage_gpt_61_sol_v1 · rescored 77adc97b: 0.538triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.538triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0.538triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0.538triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0.538triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.538triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.538triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0.538triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0.538triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0.538triage_local_dgx_nemotron_35_super_120b_v2: 0.554triage_local_mac_gpt_oss_20b_v2: 0.554triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0.554triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.554triage_sonnet_55_v1 · rescored 3d3a6afd: 0.554triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0.554triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0.554triage_sonnet_55_v1 · rescored 5f2c057c: 0.554triage_sonnet_55_v1 · rescored 77adc97b: 0.554triage_sonnet_55_v1 · rescored cd35dc1f: 0.554triage_kimi_k31_pro_v2: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1: 0.569triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.569triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.569triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.569triage_deepseek_41flash_v1 · rescored 77adc97b: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.569triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.569triage_local_dgx_ministral_3_14b_v1: 0.585triage_local_dgx_nemotron_35_lightning_30b_v2: 0.585triage_local_mac_gemma4_31b_v2: 0.585triage_local_mac_qwen3_30b_a3b_v1: 0.585triage_local_x3_nemotron_35_lightning_30b_v2: 0.585triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0.585triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.585triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0.585triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.585triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.585triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0.585triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.585triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0.585triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.585triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.585triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0.585triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.585triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0.585triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.585triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.59triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.59triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.59triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.59triage_local_dgx_gpt_oss_20b_v2: 0.6triage_local_mac_nemotron_35_lightning_30b_v2: 0.6triage_local_mac_phi4_14b_v1: 0.6triage_local_x3_ministral_3_14b_v1: 0.6triage_baseline_pro_v1 · rescored 3d3a6afd: 0.6triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.6triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.6triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0.6triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0.6triage_baseline_pro_v1 · rescored 5f2c057c: 0.6triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0.6triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.6triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0.6triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0.6triage_baseline_pro_v1 · rescored 77adc97b: 0.6triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0.6triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0.6triage_baseline_pro_v1 · rescored cd35dc1f: 0.6triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0.6triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0.6triage_local_dgx_qwen38_27b_v1: 0.615triage_local_dgx_qwen3_30b_a3b_v1: 0.615triage_local_mac_ministral_3_14b_v1: 0.615triage_local_x3_phi4_14b_v1: 0.615triage_mistral_large_4_v2: 0.615triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.615triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0.615triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.615triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0.615triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.615triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0.615triage_mistral_large_4_v1 · rescored 5f2c057c: 0.615triage_mistral_large_4_v2 · rescored 5f2c057c: 0.615triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0.615triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.615triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0.615triage_mistral_large_4_v1 · rescored 77adc97b: 0.615triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0.615triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.615triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0.615triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0.615triage_mistral_large_4_v1 · rescored cd35dc1f: 0.615triage_hosted_qwen35_122b_v2: 0.631triage_local_x3_gpt_oss_20b_v2: 0.631triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.631triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.631triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.631triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.631triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.631triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0.631triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.631triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.631triage_stub_dismiss_v1: 0.646triage_stub_dismiss_v1 · rescored 3d3a6afd: 0.646triage_stub_dismiss_v1 · rescored 5f2c057c: 0.646triage_stub_dismiss_v1 · rescored 77adc97b: 0.646triage_stub_dismiss_v1 · rescored cd35dc1f: 0.646triage_deepseek_41flash_v2: 0.662triage_local_x3_qwen38_27b_v1: 0.662triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.662triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0.662triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.662triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0.662triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0.662triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0.662triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0.662triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0.662triage_local_dgx_mistral_medium_35_128b_v1: 0.723triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 0.723triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 0.723triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 0.723triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 0.723triage_local_mac_qwen38_27b_v1: 0.738triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 0.738triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 0.738triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 0.738triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 0.738triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 0.738triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 0.769triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 0.769triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 0.769triage_baseline_pro_noreason_v1: 0.785triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 0.785triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 0.785triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 0.785triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 0.785triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 0.785triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 0.785triage_local_dgx_qwen35_122b_v1: 0.8triage_local_dgx_qwen38_27b_v2: 0.8triage_local_mac_qwen38_27b_v2: 0.8triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 0.8triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 0.8triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 0.8triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 0.8triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 0.8triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 0.8triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 0.8triage_local_x3_qwen38_27b_v2: 0.892triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 0.892triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 0.892triage_local_dgx_llama4_scout_v1: 0.923triage_local_dgx_qwen35_122b_v2: 0.923triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 0.923triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 0.923triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 0.923triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 0.923triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 0.923triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 0.9230.369 · better0.923

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean triage_cost_p3itemsscored against
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v10.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v10.36965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd0.36965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c0.36965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b0.36965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f0.36965gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v10.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v10.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v10.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v10.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v20.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v10.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.43165gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd0.44665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b0.44665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f0.44665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v20.46265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v10.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v10.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v10.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v20.46265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b0.46265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b0.46265gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f0.46265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f0.46265gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v10.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v10.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats0.47765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×30.48765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×30.48765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×30.48765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×30.48765gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×30.49265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c0.49265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×30.49265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b0.49265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×30.49265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f0.49265gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×30.49265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×30.50365gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×30.50365gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×30.50365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v20.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v20.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v10.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v20.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v20.52365gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd0.52365gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v20.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v20.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v10.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v10.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v10.53865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v20.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v20.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b0.55465gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f0.55465gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v20.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v10.56965gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd0.56965gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.56965gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v10.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v20.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v20.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v10.58565gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v20.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×30.59065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×30.59065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×30.59065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×30.59065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v20.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v20.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v10.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v10.60065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd0.60065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd0.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd0.60065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b0.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b0.60065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f0.60065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f0.60065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v10.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v10.61565gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v10.61565gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v10.61565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v20.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.61565gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd0.61565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd0.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c0.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c0.61565gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c0.61565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c0.61565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c0.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b0.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b0.61565gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b0.61565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b0.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f0.61565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f0.61565gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f0.61565gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f0.61565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f0.61565gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v20.63165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v20.63165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd0.63165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd0.63165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd0.63165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b0.63165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f0.63165gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v10.64665gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd0.64665gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c0.64665gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b0.64665gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f0.64665gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v20.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v10.66265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd0.66265gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd0.66265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f0.66265gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v10.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b0.72365gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f0.72365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v10.73865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c0.73865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c0.73865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b0.73865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b0.73865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f0.73865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c0.76965gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b0.76965gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f0.76965gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v10.78565gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd0.78565gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c0.78565gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c0.78565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c0.78565gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b0.78565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b0.78565gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v10.80065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v20.80065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v20.80065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd0.80065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd0.80065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c0.80065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c0.80065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c0.80065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b0.80065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f0.80065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v20.89265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd0.89265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c0.89265gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v10.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v20.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd0.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd0.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c0.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c0.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b0.92365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f0.92365gold/orrery_triage_v1_labelled gold