eval-harness

metric

false_dismiss

better
lower
kind
quality: measured against a reference or by a judge
decides
no goal
weighed by
no goal
recorded on
orrery_triage_v1_labelled

01what it measures, and how

1 when a real problem (the label: file or escalate) was dismissed or cleaned up: a miss.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"false_dismiss": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_local_dgx_llama31_8b_v1: 0triage_local_mac_llama31_8b_v1: 0triage_local_x3_llama31_8b_v1: 0triage_stub_escalate_v1: 0triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0triage_stub_escalate_v1 · rescored 3d3a6afd: 0triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0triage_stub_escalate_v1 · rescored 5f2c057c: 0triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0triage_stub_escalate_v1 · rescored 77adc97b: 0triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0triage_stub_escalate_v1 · rescored cd35dc1f: 0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0.029triage_local_x3_gpt_oss_20b_v1: 0.029triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0.029triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0.029triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0.029triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0.029triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0.029triage_local_dgx_gpt_oss_20b_v1: 0.059triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0.059triage_gemini_35_flash_lite_v1: 0.088triage_gpt_6_luna_v1: 0.088triage_gpt_6_luna_v2: 0.088triage_local_dgx_nemotron_35_bolt_e4b_v1: 0.088triage_local_dgx_nemotron_35_lightning_30b_v2: 0.088triage_local_mac_gpt_oss_20b_v1: 0.088triage_local_mac_nemotron_35_lightning_30b_v2: 0.088triage_local_x3_nemotron_35_lightning_30b_v1: 0.088triage_local_x3_nemotron_35_lightning_30b_v2: 0.088triage_gpt_61_sol_v1 · repeats ×3: 0.088triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.088triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.088triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.088triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.088triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.088triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.088triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.088triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.088triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.088triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.088triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.088triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.088triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.088triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.088triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.088triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.088triage_gpt_6_luna_v1 · rescored 77adc97b: 0.088triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.088triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.088triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.088triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.088triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.088triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.088triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0.088triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.088triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.088triage_gemini_4_argon_v1: 0.118triage_gpt_61_sol_v1: 0.118triage_haiku_55_v1: 0.118triage_hosted_gpt_oss_120b_v2: 0.118triage_kimi_k31_pro_v1: 0.118triage_local_dgx_gemma4_12b_noreason_v1: 0.118triage_local_dgx_gpt_oss_20b_v2: 0.118triage_local_dgx_nemotron_35_bolt_e4b_v2: 0.118triage_local_dgx_nemotron_35_lightning_30b_v1: 0.118triage_local_dgx_nemotron_35_super_120b_v2: 0.118triage_local_dgx_phi4_14b_v1: 0.118triage_local_mac_gemma4_12b_noreason_v1: 0.118triage_local_mac_nemotron_35_bolt_e4b_v2: 0.118triage_local_mac_nemotron_35_lightning_30b_v1: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1: 0.118triage_local_x3_nemotron_35_bolt_e4b_v2: 0.118triage_gpt_6_luna_v1 · repeats ×3: 0.118triage_haiku_55_v1 · repeats ×3: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0.118triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.118triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.118triage_haiku_55_v1 · rescored 3d3a6afd: 0.118triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.118triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.118triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.118triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.118triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0.118triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0.118triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.118triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.118triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.118triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.118triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.118triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.118triage_haiku_55_v1 · rescored 5f2c057c: 0.118triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.118triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.118triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.118triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0.118triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.118triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.118triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0.118triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0.118triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.118triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.118triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.118triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.118triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.118triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0.118triage_gemini_4_argon_v1 · rescored 77adc97b: 0.118triage_gpt_61_sol_v1 · rescored 77adc97b: 0.118triage_haiku_55_v1 · rescored 77adc97b: 0.118triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.118triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.118triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.118triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0.118triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.118triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.118triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.118triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0.118triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.118triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.118triage_haiku_55_v1 · rescored cd35dc1f: 0.118triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.118triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.118triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.118triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0.118triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.118triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.118triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.118triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.118triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0.118triage_hosted_gpt_oss_120b_v1: 0.147triage_kimi_k31_pro_v2: 0.147triage_local_dgx_gemma4_31b_v1: 0.147triage_local_dgx_gpt_oss_120b_v1: 0.147triage_local_dgx_gpt_oss_120b_v2: 0.147triage_local_dgx_nemotron_35_super_120b_v1: 0.147triage_local_mac_gemma4_31b_v1: 0.147triage_local_mac_gpt_oss_20b_v2: 0.147triage_local_mac_phi4_14b_v1: 0.147triage_local_x3_gemma4_31b_v1: 0.147triage_local_x3_phi4_14b_v1: 0.147triage_sonnet_55_v1: 0.147triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.147triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.147triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.147triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0.147triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0.147triage_sonnet_55_v1 · rescored 3d3a6afd: 0.147triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.147triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.147triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0.147triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0.147triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0.147triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0.147triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0.147triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0.147triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0.147triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0.147triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0.147triage_sonnet_55_v1 · rescored 5f2c057c: 0.147triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.147triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0.147triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0.147triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0.147triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0.147triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0.147triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0.147triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0.147triage_sonnet_55_v1 · rescored 77adc97b: 0.147triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.147triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0.147triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0.147triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0.147triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0.147triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0.147triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0.147triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0.147triage_sonnet_55_v1 · rescored cd35dc1f: 0.147triage_baseline_pro_v1 · repeats ×3: 0.167triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.167triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.167triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.167triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.167triage_baseline_pro_v1: 0.176triage_deepseek_41flash_v1: 0.176triage_hosted_gemma4_31b_v1: 0.176triage_hosted_gemma4_31b_v2: 0.176triage_hosted_qwen3_30b_a3b_v1: 0.176triage_local_dgx_gemma4_31b_v2: 0.176triage_local_mac_nemotron_35_bolt_e4b_v1: 0.176triage_local_x3_gemma4_31b_v2: 0.176triage_local_x3_gpt_oss_20b_v2: 0.176triage_mistral_large_4_v1: 0.176triage_mistral_large_4_v2: 0.176triage_baseline_pro_v1 · rescored 3d3a6afd: 0.176triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.176triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.176triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0.176triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0.176triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.176triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.176triage_baseline_pro_v1 · rescored 5f2c057c: 0.176triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.176triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.176triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.176triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.176triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0.176triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.176triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0.176triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0.176triage_mistral_large_4_v1 · rescored 5f2c057c: 0.176triage_mistral_large_4_v2 · rescored 5f2c057c: 0.176triage_baseline_pro_v1 · rescored 77adc97b: 0.176triage_deepseek_41flash_v1 · rescored 77adc97b: 0.176triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.176triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.176triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.176triage_mistral_large_4_v1 · rescored 77adc97b: 0.176triage_baseline_pro_v1 · rescored cd35dc1f: 0.176triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.176triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.176triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.176triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.176triage_mistral_large_4_v1 · rescored cd35dc1f: 0.176triage_local_dgx_qwen38_27b_v1: 0.206triage_local_mac_gemma4_31b_v2: 0.206triage_local_mac_ministral_3_14b_v1: 0.206triage_local_x3_ministral_3_14b_v1: 0.206triage_local_x3_qwen3_30b_a3b_v1: 0.206triage_stub_dismiss_v1: 0.206triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0.206triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0.206triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.206triage_stub_dismiss_v1 · rescored 3d3a6afd: 0.206triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0.206triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0.206triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0.206triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.206triage_stub_dismiss_v1 · rescored 5f2c057c: 0.206triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0.206triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0.206triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.206triage_stub_dismiss_v1 · rescored 77adc97b: 0.206triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0.206triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0.206triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0.206triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.206triage_stub_dismiss_v1 · rescored cd35dc1f: 0.206triage_deepseek_41flash_v2: 0.235triage_hosted_mistral_medium_35_v1: 0.235triage_hosted_qwen35_122b_v2: 0.235triage_local_dgx_ministral_3_14b_v1: 0.235triage_local_mac_qwen3_30b_a3b_v1: 0.235triage_local_x3_qwen38_27b_v1: 0.235triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.235triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.235triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.235triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0.235triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0.235triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.235triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.235triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.235triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.235triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0.235triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0.235triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.235triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0.235triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.235triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0.235triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0.235triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.235triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0.235triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.235triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0.235triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.235triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0.235triage_local_dgx_mistral_medium_35_128b_v1: 0.265triage_local_dgx_qwen3_30b_a3b_v1: 0.265triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 0.265triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.265triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 0.265triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 0.265triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.265triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 0.265triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 0.265triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.265triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 0.265triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.265triage_hosted_qwen35_122b_v1: 0.294triage_local_mac_qwen38_27b_v1: 0.294triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 0.294triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 0.294triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 0.294triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 0.294triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 0.294triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 0.294triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 0.294triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 0.294triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 0.294triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 0.294triage_local_dgx_llama4_scout_v1: 0.324triage_local_dgx_qwen35_122b_v1: 0.324triage_local_dgx_qwen38_27b_v2: 0.324triage_local_mac_qwen38_27b_v2: 0.324triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 0.324triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 0.324triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 0.324triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 0.324triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 0.324triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 0.324triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 0.324triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 0.324triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 0.324triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 0.324triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 0.324triage_baseline_pro_noreason_v1: 0.353triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 0.353triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 0.353triage_local_x3_qwen38_27b_v2: 0.382triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 0.382triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 0.382triage_local_dgx_qwen35_122b_v2: 0.412triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 0.412triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 0.4120 · better0.412

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean false_dismissitemsscored against
triage_local_dgx_llama31_8b_v10.00034gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v10.00034gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v10.00034gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v10.00034gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd0.00034gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd0.00034gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd0.00034gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd0.00034gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c0.00034gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c0.00034gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c0.00034gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c0.00034gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b0.00034gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b0.00034gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b0.00034gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b0.00034gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f0.00034gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f0.00034gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f0.00034gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f0.00034gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v10.02934gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v10.02934gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd0.02934gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c0.02934gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b0.02934gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f0.02934gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f0.02934gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v10.05934gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f0.05934gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v10.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v10.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v20.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v10.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v20.08834gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v10.08834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v20.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v10.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v20.08834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×30.08834gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd0.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.08834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×30.08834gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×30.08834gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b0.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×30.08834gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f0.08834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.08834gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f0.08834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.08834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×30.08834gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v10.11834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v10.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v10.11834gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v20.11834gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v20.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v20.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v20.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v20.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v10.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v20.11834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats0.11834gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats0.11834gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats0.11834gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.11834gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×30.11834gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats0.11834gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v10.14734gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v20.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v10.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v10.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v20.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v10.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v10.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v20.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v10.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v10.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v10.14734gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v10.14734gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd0.14734gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd0.14734gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c0.14734gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b0.14734gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f0.14734gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×30.16734gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×30.16734gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×30.16734gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×30.16734gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×30.16734gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v10.17634gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v10.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v10.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v20.17634gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v10.17634gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v20.17634gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v10.17634gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v20.17634gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v20.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v10.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v20.17634gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd0.17634gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c0.17634gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b0.17634gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b0.17634gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b0.17634gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f0.17634gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f0.17634gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f0.17634gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.17634gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f0.17634gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v10.20634gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v20.20634gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v10.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v10.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v10.20634gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v10.20634gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.20634gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd0.20634gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c0.20634gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c0.20634gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c0.20634gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b0.20634gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b0.20634gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f0.20634gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f0.20634gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f0.20634gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f0.20634gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v20.23534gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v10.23534gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v20.23534gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v10.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v10.23534gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v10.23534gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd0.23534gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd0.23534gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd0.23534gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.23534gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c0.23534gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b0.23534gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b0.23534gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b0.23534gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f0.23534gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f0.23534gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f0.23534gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f0.23534gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v10.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v10.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f0.26534gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f0.26534gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v10.29434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v10.29434gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c0.29434gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c0.29434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c0.29434gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c0.29434gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b0.29434gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b0.29434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b0.29434gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b0.29434gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f0.29434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f0.29434gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v10.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v10.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v20.32434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v20.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd0.32434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c0.32434gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f0.32434gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f0.32434gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v10.35334gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd0.35334gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c0.35334gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v20.38234gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd0.38234gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c0.38234gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v20.41234gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd0.41234gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c0.41234gold/orrery_triage_v1_labelled gold