eval-harness

metric

false_alarm

better
lower
kind
quality: measured against a reference or by a judge
decides
no goal
weighed by
Triage decisions on Orrery's error signals
recorded on
orrery_triage_v1_labelled

01what it measures, and how

1 when noise (the label: dismiss or cleanup) was filed or escalated.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"false_alarm": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_local_x3_qwen3_30b_a3b_v1: 0.387triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.387triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.387triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.387triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.387triage_local_dgx_qwen3_30b_a3b_v1: 0.419triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.419triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.419triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.419triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.419triage_local_dgx_ministral_3_14b_v1: 0.452triage_local_mac_qwen3_30b_a3b_v1: 0.452triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0.452triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.452triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0.452triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.452triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0.452triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.452triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0.452triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.452triage_baseline_pro_noreason_v1: 0.484triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 0.484triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 0.484triage_hosted_qwen3_30b_a3b_v1: 0.516triage_local_x3_gemma4_31b_v2: 0.516triage_local_x3_nemotron_35_bolt_e4b_v2: 0.516triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.516triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0.516triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.516triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.516triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0.516triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.516triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.516triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.516triage_hosted_gemma4_31b_v1: 0.548triage_hosted_gemma4_31b_v2: 0.548triage_hosted_mistral_medium_35_v1: 0.548triage_hosted_qwen35_122b_v2: 0.548triage_local_dgx_gemma4_31b_v2: 0.548triage_local_mac_gemma4_31b_v2: 0.548triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.548triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.548triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.548triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0.548triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0.548triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.548triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.548triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.548triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.548triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0.548triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0.548triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.548triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.548triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.548triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.548triage_haiku_55_v1: 0.581triage_local_dgx_gemma4_12b_noreason_v1: 0.581triage_local_dgx_gpt_oss_120b_v2: 0.581triage_local_dgx_qwen35_122b_v2: 0.581triage_local_mac_gemma4_12b_noreason_v1: 0.581triage_local_mac_nemotron_35_bolt_e4b_v2: 0.581triage_local_mac_qwen38_27b_v1: 0.581triage_local_x3_ministral_3_14b_v1: 0.581triage_haiku_55_v1 · rescored 3d3a6afd: 0.581triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.581triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.581triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 0.581triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.581triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.581triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0.581triage_haiku_55_v1 · rescored 5f2c057c: 0.581triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.581triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0.581triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 0.581triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.581triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.581triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 0.581triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0.581triage_haiku_55_v1 · rescored 77adc97b: 0.581triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.581triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.581triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 0.581triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0.581triage_haiku_55_v1 · rescored cd35dc1f: 0.581triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.581triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.581triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 0.581triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0.581triage_deepseek_41flash_v1: 0.613triage_deepseek_41flash_v2: 0.613triage_hosted_gpt_oss_120b_v1: 0.613triage_local_dgx_gemma4_31b_v1: 0.613triage_local_dgx_nemotron_35_bolt_e4b_v1: 0.613triage_local_dgx_nemotron_35_super_120b_v1: 0.613triage_local_dgx_qwen35_122b_v1: 0.613triage_local_dgx_qwen38_27b_v1: 0.613triage_local_dgx_qwen38_27b_v2: 0.613triage_local_mac_gemma4_31b_v1: 0.613triage_local_mac_ministral_3_14b_v1: 0.613triage_local_mac_nemotron_35_bolt_e4b_v1: 0.613triage_local_mac_qwen38_27b_v2: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1: 0.613triage_local_x3_nemotron_35_lightning_30b_v1: 0.613triage_local_x3_qwen38_27b_v1: 0.613triage_local_x3_qwen38_27b_v2: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0.613triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.613triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 0.613triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0.613triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 0.613triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 0.613triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.613triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.613triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.613triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0.613triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.613triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0.613triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 0.613triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0.613triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 0.613triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0.613triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0.613triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.613triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.613triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.613triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0.613triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0.613triage_deepseek_41flash_v1 · rescored 77adc97b: 0.613triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.613triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0.613triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.613triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0.613triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 0.613triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0.613triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0.613triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0.613triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.613triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.613triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0.613triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.613triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.613triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0.613triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.613triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0.613triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 0.613triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0.613triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0.613triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0.613triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.613triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.613triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0.613triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0.613triage_haiku_55_v1 · repeats ×3: 0.634triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.634triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.634triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.634triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.634triage_gemini_35_flash_lite_v1: 0.645triage_hosted_qwen35_122b_v1: 0.645triage_local_dgx_gpt_oss_120b_v1: 0.645triage_local_dgx_mistral_medium_35_128b_v1: 0.645triage_local_x3_gemma4_31b_v1: 0.645triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.645triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 0.645triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.645triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 0.645triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0.645triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 0.645triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0.645triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.645triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 0.645triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0.645triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 0.645triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0.645triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.645triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 0.645triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0.645triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 0.645triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0.645triage_gpt_6_luna_v1 · repeats ×3: 0.667triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.667triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.667triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.667triage_baseline_pro_v1: 0.677triage_gpt_6_luna_v2: 0.677triage_hosted_gpt_oss_120b_v2: 0.677triage_local_dgx_gpt_oss_20b_v1: 0.677triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0.677triage_local_mac_gpt_oss_20b_v1: 0.677triage_local_mac_gpt_oss_20b_v2: 0.677triage_local_x3_gpt_oss_20b_v1: 0.677triage_sonnet_55_v1: 0.677triage_stub_dismiss_v1: 0.677triage_baseline_pro_v1 · rescored 3d3a6afd: 0.677triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.677triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.677triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0.677triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.677triage_sonnet_55_v1 · rescored 3d3a6afd: 0.677triage_stub_dismiss_v1 · rescored 3d3a6afd: 0.677triage_baseline_pro_v1 · rescored 5f2c057c: 0.677triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.677triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.677triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 0.677triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0.677triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 0.677triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0.677triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 0.677triage_sonnet_55_v1 · rescored 5f2c057c: 0.677triage_stub_dismiss_v1 · rescored 5f2c057c: 0.677triage_baseline_pro_v1 · rescored 77adc97b: 0.677triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 0.677triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0.677triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 0.677triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 0.677triage_sonnet_55_v1 · rescored 77adc97b: 0.677triage_stub_dismiss_v1 · rescored 77adc97b: 0.677triage_baseline_pro_v1 · rescored cd35dc1f: 0.677triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0.677triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0.677triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0.677triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0.677triage_sonnet_55_v1 · rescored cd35dc1f: 0.677triage_stub_dismiss_v1 · rescored cd35dc1f: 0.677triage_baseline_pro_v1 · repeats ×3: 0.688triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.688triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.688triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.688triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.688triage_gemini_4_argon_v1: 0.71triage_kimi_k31_pro_v1: 0.71triage_kimi_k31_pro_v2: 0.71triage_local_dgx_nemotron_35_bolt_e4b_v2: 0.71triage_local_dgx_nemotron_35_lightning_30b_v1: 0.71triage_local_mac_nemotron_35_lightning_30b_v1: 0.71triage_mistral_large_4_v1: 0.71triage_mistral_large_4_v2: 0.71triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.71triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.71triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.71triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.71triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.71triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.71triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.71triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.71triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.71triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.71triage_mistral_large_4_v1 · rescored 5f2c057c: 0.71triage_mistral_large_4_v2 · rescored 5f2c057c: 0.71triage_gemini_4_argon_v1 · rescored 77adc97b: 0.71triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.71triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.71triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.71triage_mistral_large_4_v1 · rescored 77adc97b: 0.71triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.71triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.71triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.71triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.71triage_mistral_large_4_v1 · rescored cd35dc1f: 0.71triage_gpt_61_sol_v1: 0.742triage_gpt_6_luna_v1: 0.742triage_local_dgx_phi4_14b_v1: 0.742triage_local_x3_gpt_oss_20b_v2: 0.742triage_gpt_61_sol_v1 · repeats ×3: 0.742triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.742triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0.742triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.742triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.742triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.742triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.742triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0.742triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0.742triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.742triage_gpt_61_sol_v1 · rescored 77adc97b: 0.742triage_gpt_6_luna_v1 · rescored 77adc97b: 0.742triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0.742triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.742triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.742triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.742triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0.742triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.742triage_local_dgx_nemotron_35_super_120b_v2: 0.774triage_local_mac_phi4_14b_v1: 0.774triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0.774triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0.774triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0.774triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0.774triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0.774triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0.774triage_local_x3_phi4_14b_v1: 0.806triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0.806triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0.806triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0.806triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0.806triage_local_dgx_gpt_oss_20b_v2: 0.871triage_local_dgx_llama31_8b_v1: 0.871triage_local_dgx_llama4_scout_v1: 0.871triage_local_mac_llama31_8b_v1: 0.871triage_local_x3_llama31_8b_v1: 0.871triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.871triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0.871triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 0.871triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0.871triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0.871triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0.871triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0.871triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 0.871triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0.871triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0.871triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0.871triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 0.871triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0.871triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0.871triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0.871triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 0.871triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0.871triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0.871triage_local_dgx_nemotron_35_lightning_30b_v2: 0.935triage_local_x3_nemotron_35_lightning_30b_v2: 0.935triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.935triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.935triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.935triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.935triage_local_mac_nemotron_35_lightning_30b_v2: 0.968triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.968triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.968triage_stub_escalate_v1: 1triage_stub_escalate_v1 · rescored 3d3a6afd: 1triage_stub_escalate_v1 · rescored 5f2c057c: 1triage_stub_escalate_v1 · rescored 77adc97b: 1triage_stub_escalate_v1 · rescored cd35dc1f: 10.387 · better1

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean false_alarmitemsscored against
triage_local_x3_qwen3_30b_a3b_v10.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f0.38731gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v10.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v10.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v10.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v10.48431gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd0.48431gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c0.48431gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v10.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v20.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v20.51631gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd0.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.51631gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c0.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c0.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.51631gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b0.51631gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f0.51631gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v10.54831gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v20.54831gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v10.54831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v20.54831gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v20.54831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v20.54831gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd0.54831gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd0.54831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c0.54831gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c0.54831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f0.54831gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f0.54831gold/orrery_triage_v1_labelled gold
triage_haiku_55_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v20.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v20.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v20.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v10.58131gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd0.58131gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v10.61331gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v20.61331gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v20.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v20.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v10.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v20.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats0.61331gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd0.61331gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats0.61331gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats0.61331gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f0.61331gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats0.61331gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×30.63431gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×30.63431gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×30.63431gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×30.63431gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×30.63431gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v10.64531gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v10.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v10.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v10.64531gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v10.64531gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd0.64531gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c0.64531gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c0.64531gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c0.64531gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b0.64531gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b0.64531gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b0.64531gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f0.64531gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f0.64531gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f0.64531gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f0.64531gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×30.66731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×30.66731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×30.66731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×30.66731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v10.67731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v20.67731gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v20.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v10.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v10.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v10.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v20.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v10.67731gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v10.67731gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v10.67731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×30.68831gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×30.68831gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×30.68831gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×30.68831gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×30.68831gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v10.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v10.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v20.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v20.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v10.71031gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v10.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v10.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v20.71031gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd0.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd0.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd0.71031gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c0.71031gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b0.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b0.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.71031gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b0.71031gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f0.71031gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f0.71031gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.71031gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.71031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f0.71031gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v10.74231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v10.74231gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v10.74231gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v20.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×30.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd0.74231gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd0.74231gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×30.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c0.74231gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c0.74231gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×30.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b0.74231gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×30.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f0.74231gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f0.74231gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×30.74231gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v20.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v10.77431gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd0.77431gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f0.77431gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v10.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd0.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c0.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b0.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f0.80631gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v20.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v10.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v10.87131gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v10.87131gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v10.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd0.87131gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd0.87131gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c0.87131gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c0.87131gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b0.87131gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b0.87131gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f0.87131gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f0.87131gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v20.93531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v20.93531gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.93531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.93531gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.93531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.93531gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v20.96831gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.96831gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.96831gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v11.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd1.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c1.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b1.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f1.00031gold/orrery_triage_v1_labelled gold