eval-harness

metric

model_false_alarm

better
lower
kind
quality: measured against a reference or by a judge
decides
no goal
weighed by
no goal
recorded on
orrery_triage_v1_labelled

01what it measures, and how

false_alarm of the model's own decision, before the fleet's gates.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"model_false_alarm": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_stub_dismiss_v1: 0triage_stub_dismiss_v1 · rescored 3d3a6afd: 0triage_stub_dismiss_v1 · rescored 5f2c057c: 0triage_stub_dismiss_v1 · rescored 77adc97b: 0triage_stub_dismiss_v1 · rescored cd35dc1f: 0triage_deepseek_41flash_v2: 0.065triage_local_dgx_ministral_3_14b_v1: 0.065triage_local_mac_ministral_3_14b_v1: 0.065triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.065triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0.065triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0.065triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.065triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0.065triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0.065triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0.065triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0.065triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0.065triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0.065triage_local_x3_ministral_3_14b_v1: 0.097triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0.097triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0.097triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0.097triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0.097triage_baseline_pro_noreason_v1: 0.129triage_hosted_qwen35_122b_v2: 0.129triage_local_dgx_qwen35_122b_v2: 0.129triage_local_x3_gemma4_31b_v2: 0.129triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 0.129triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.129triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 0.129triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0.129triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 0.129triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.129triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 0.129triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0.129triage_hosted_gemma4_31b_v1: 0.161triage_hosted_gemma4_31b_v2: 0.161triage_local_dgx_gemma4_31b_v2: 0.161triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.161triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0.161triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.161triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.161triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0.161triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.161triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.161triage_deepseek_41flash_v1: 0.194triage_local_mac_gemma4_31b_v2: 0.194triage_local_mac_qwen3_30b_a3b_v1: 0.194triage_local_x3_qwen3_30b_a3b_v1: 0.194triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0.194triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.194triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.194triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.194triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0.194triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.194triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.194triage_deepseek_41flash_v1 · rescored 77adc97b: 0.194triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.194triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.194triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.194triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.194triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.194triage_gemini_35_flash_lite_v1: 0.226triage_gemini_4_argon_v1: 0.226triage_local_dgx_qwen35_122b_v1: 0.226triage_local_dgx_qwen3_30b_a3b_v1: 0.226triage_sonnet_55_v1: 0.226triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.226triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.226triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.226triage_sonnet_55_v1 · rescored 3d3a6afd: 0.226triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.226triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.226triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 0.226triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.226triage_sonnet_55_v1 · rescored 5f2c057c: 0.226triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.226triage_gemini_4_argon_v1 · rescored 77adc97b: 0.226triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 0.226triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.226triage_sonnet_55_v1 · rescored 77adc97b: 0.226triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.226triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.226triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 0.226triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.226triage_sonnet_55_v1 · rescored cd35dc1f: 0.226triage_hosted_qwen3_30b_a3b_v1: 0.258triage_local_dgx_qwen38_27b_v1: 0.258triage_local_dgx_qwen38_27b_v2: 0.258triage_local_mac_gemma4_12b_noreason_v1: 0.258triage_local_mac_qwen38_27b_v1: 0.258triage_local_x3_qwen38_27b_v2: 0.258triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.258triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 0.258triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.258triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 0.258triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.258triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0.258triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 0.258triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.258triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 0.258triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 0.258triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.258triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0.258triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.258triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 0.258triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.258triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0.258triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.258triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 0.258triage_local_dgx_gemma4_12b_noreason_v1: 0.29triage_local_x3_qwen38_27b_v1: 0.29triage_mistral_large_4_v2: 0.29triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.29triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.29triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.29triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0.29triage_mistral_large_4_v2 · rescored 5f2c057c: 0.29triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.29triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0.29triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.29triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0.29triage_hosted_qwen35_122b_v1: 0.323triage_local_mac_qwen38_27b_v2: 0.323triage_mistral_large_4_v1: 0.323triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 0.323triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 0.323triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 0.323triage_mistral_large_4_v1 · rescored 5f2c057c: 0.323triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 0.323triage_mistral_large_4_v1 · rescored 77adc97b: 0.323triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 0.323triage_mistral_large_4_v1 · rescored cd35dc1f: 0.323triage_baseline_pro_v1: 0.355triage_haiku_55_v1: 0.355triage_kimi_k31_pro_v2: 0.355triage_local_dgx_gemma4_31b_v1: 0.355triage_local_dgx_nemotron_35_bolt_e4b_v1: 0.355triage_local_mac_nemotron_35_bolt_e4b_v2: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0.355triage_baseline_pro_v1 · rescored 3d3a6afd: 0.355triage_haiku_55_v1 · rescored 3d3a6afd: 0.355triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.355triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.355triage_baseline_pro_v1 · rescored 5f2c057c: 0.355triage_haiku_55_v1 · rescored 5f2c057c: 0.355triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.355triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0.355triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.355triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0.355triage_baseline_pro_v1 · rescored 77adc97b: 0.355triage_haiku_55_v1 · rescored 77adc97b: 0.355triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0.355triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0.355triage_baseline_pro_v1 · rescored cd35dc1f: 0.355triage_haiku_55_v1 · rescored cd35dc1f: 0.355triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0.355triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.355triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0.355triage_baseline_pro_v1 · repeats ×3: 0.366triage_haiku_55_v1 · repeats ×3: 0.366triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.366triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.366triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.366triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.366triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.366triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.366triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.366triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.366triage_hosted_gpt_oss_120b_v1: 0.387triage_local_dgx_gpt_oss_120b_v2: 0.387triage_local_mac_gemma4_31b_v1: 0.387triage_local_x3_gemma4_31b_v1: 0.387triage_local_x3_nemotron_35_bolt_e4b_v2: 0.387triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.387triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.387triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.387triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0.387triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0.387triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0.387triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.387triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.387triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0.387triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0.387triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.387triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0.387triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0.387triage_gpt_6_luna_v1 · repeats ×3: 0.409triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.409triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.409triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.409triage_hosted_gpt_oss_120b_v2: 0.419triage_hosted_mistral_medium_35_v1: 0.419triage_kimi_k31_pro_v1: 0.419triage_local_dgx_mistral_medium_35_128b_v1: 0.419triage_local_dgx_nemotron_35_super_120b_v1: 0.419triage_local_x3_nemotron_35_lightning_30b_v1: 0.419triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.419triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.419triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 0.419triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.419triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.419triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.419triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 0.419triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0.419triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.419triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.419triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.419triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 0.419triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0.419triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.419triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.419triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.419triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 0.419triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0.419triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.419triage_gpt_6_luna_v2: 0.452triage_local_dgx_gpt_oss_120b_v1: 0.452triage_local_dgx_nemotron_35_lightning_30b_v1: 0.452triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0.452triage_local_mac_gpt_oss_20b_v2: 0.452triage_local_mac_nemotron_35_bolt_e4b_v1: 0.452triage_local_mac_nemotron_35_lightning_30b_v1: 0.452triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.452triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0.452triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.452triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.452triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0.452triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.452triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0.452triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0.452triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.452triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.452triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0.452triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.452triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0.452triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.452triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.452triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0.452triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.452triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0.452triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.452triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.452triage_gpt_6_luna_v1: 0.484triage_local_dgx_nemotron_35_super_120b_v2: 0.484triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0.484triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.484triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0.484triage_gpt_6_luna_v1 · rescored 77adc97b: 0.484triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.484triage_gpt_61_sol_v1 · repeats ×3: 0.505triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.505triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.505triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.505triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.505triage_gpt_61_sol_v1: 0.516triage_local_dgx_nemotron_35_bolt_e4b_v2: 0.516triage_local_x3_gpt_oss_20b_v2: 0.516triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.516triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.516triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.516triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.516triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.516triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0.516triage_gpt_61_sol_v1 · rescored 77adc97b: 0.516triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.516triage_local_dgx_gpt_oss_20b_v1: 0.581triage_local_mac_gpt_oss_20b_v1: 0.581triage_local_x3_gpt_oss_20b_v1: 0.581triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 0.581triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 0.581triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 0.581triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 0.581triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 0.581triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 0.581triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0.581triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0.581triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0.581triage_local_dgx_gpt_oss_20b_v2: 0.677triage_local_dgx_phi4_14b_v1: 0.677triage_local_mac_phi4_14b_v1: 0.677triage_local_x3_phi4_14b_v1: 0.677triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.677triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0.677triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0.677triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0.677triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0.677triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0.677triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0.677triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0.677triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0.677triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0.677triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0.677triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0.677triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0.677triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0.677triage_local_mac_llama31_8b_v1: 0.774triage_local_x3_llama31_8b_v1: 0.774triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0.774triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0.774triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0.774triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0.774triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0.774triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0.774triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0.774triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0.774triage_local_dgx_nemotron_35_lightning_30b_v2: 0.806triage_local_mac_nemotron_35_lightning_30b_v2: 0.806triage_local_x3_nemotron_35_lightning_30b_v2: 0.806triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.806triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.806triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.806triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.806triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.806triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.806triage_local_dgx_llama31_8b_v1: 0.839triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0.839triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0.839triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0.839triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0.839triage_local_dgx_llama4_scout_v1: 0.871triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 0.871triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 0.871triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 0.871triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 0.871triage_stub_escalate_v1: 1triage_stub_escalate_v1 · rescored 3d3a6afd: 1triage_stub_escalate_v1 · rescored 5f2c057c: 1triage_stub_escalate_v1 · rescored 77adc97b: 1triage_stub_escalate_v1 · rescored cd35dc1f: 10 · better1

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean model_false_alarmitemsscored against
triage_stub_dismiss_v10.00031gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd0.00031gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c0.00031gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b0.00031gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f0.00031gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v20.06531gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v10.06531gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v10.06531gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd0.06531gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd0.06531gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd0.06531gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c0.06531gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c0.06531gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c0.06531gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b0.06531gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b0.06531gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f0.06531gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f0.06531gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v10.09731gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd0.09731gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c0.09731gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b0.09731gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f0.09731gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v10.12931gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v20.12931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v20.12931gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v20.12931gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd0.12931gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd0.12931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd0.12931gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd0.12931gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c0.12931gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c0.12931gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c0.12931gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c0.12931gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v10.16131gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v20.16131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v20.16131gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd0.16131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd0.16131gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c0.16131gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c0.16131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c0.16131gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b0.16131gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f0.16131gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v10.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v20.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v10.19431gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v10.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd0.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.19431gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.19431gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c0.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c0.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c0.19431gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c0.19431gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b0.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b0.19431gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b0.19431gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f0.19431gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f0.19431gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f0.19431gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v10.22631gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v10.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v10.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v10.22631gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v10.22631gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.22631gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c0.22631gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b0.22631gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f0.22631gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f0.22631gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f0.22631gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f0.22631gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v10.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v10.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v20.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v10.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v10.25831gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v20.25831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.25831gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd0.25831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c0.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c0.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c0.25831gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c0.25831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b0.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b0.25831gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f0.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f0.25831gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f0.25831gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v10.29031gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v10.29031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v20.29031gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.29031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd0.29031gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c0.29031gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c0.29031gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c0.29031gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b0.29031gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b0.29031gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f0.29031gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f0.29031gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v10.32331gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v20.32331gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v10.32331gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd0.32331gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c0.32331gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c0.32331gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c0.32331gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b0.32331gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b0.32331gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f0.32331gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f0.32331gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v10.35531gold/orrery_triage_v1_labelled gold
triage_haiku_55_v10.35531gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v20.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v10.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v10.35531gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v20.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v10.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats0.35531gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd0.35531gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd0.35531gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd0.35531gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.35531gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats0.35531gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b0.35531gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b0.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b0.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats0.35531gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f0.35531gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f0.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f0.35531gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.35531gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats0.35531gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×30.36631gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v10.38731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v20.38731gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v10.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v10.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v20.38731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.38731gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c0.38731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c0.38731gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.38731gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b0.38731gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b0.38731gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f0.38731gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f0.38731gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f0.38731gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×30.40931gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×30.40931gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×30.40931gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×30.40931gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v20.41931gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v10.41931gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v10.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v10.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v10.41931gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v10.41931gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd0.41931gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd0.41931gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.41931gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b0.41931gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b0.41931gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.41931gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f0.41931gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f0.41931gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f0.41931gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.41931gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v20.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v10.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v10.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v10.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v20.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v10.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v10.45231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd0.45231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.45231gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v10.48431gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v20.48431gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd0.48431gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c0.48431gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c0.48431gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b0.48431gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f0.48431gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×30.50531gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×30.50531gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×30.50531gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×30.50531gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×30.50531gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v10.51631gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v20.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v20.51631gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd0.51631gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd0.51631gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c0.51631gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.51631gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c0.51631gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b0.51631gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f0.51631gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v10.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f0.58131gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v20.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v10.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v10.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v10.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b0.67731gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f0.67731gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v10.77431gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v10.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd0.77431gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c0.77431gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b0.77431gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b0.77431gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f0.77431gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f0.77431gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v20.80631gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v20.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v20.80631gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.80631gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.80631gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.80631gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.80631gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.80631gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v10.83931gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd0.83931gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c0.83931gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b0.83931gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f0.83931gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v10.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b0.87131gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f0.87131gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v11.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd1.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c1.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b1.00031gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f1.00031gold/orrery_triage_v1_labelled gold