eval-harness

metric

model_correct

better
higher
kind
quality: measured against a reference or by a judge
decides
no goal
weighed by
no goal
recorded on
orrery_triage_v1_labelled

01what it measures, and how

correct, for the model's own decision before the fleet's gates.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"model_correct": {"summary": "…", "details": "…"}}.

02on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_haiku_55_v1: 0.677triage_haiku_55_v1 · rescored 3d3a6afd: 0.677triage_haiku_55_v1 · rescored 5f2c057c: 0.677triage_haiku_55_v1 · rescored 77adc97b: 0.677triage_haiku_55_v1 · rescored cd35dc1f: 0.677triage_gpt_6_luna_v2: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0.662triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.662triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.662triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0.662triage_gpt_6_luna_v1 · repeats ×3: 0.656triage_haiku_55_v1 · repeats ×3: 0.656triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.656triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.656triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.656triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.656triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.656triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.656triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.656triage_gpt_61_sol_v1 · repeats ×3: 0.641triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.641triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.641triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.641triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.641triage_gemini_4_argon_v1: 0.631triage_gpt_61_sol_v1: 0.631triage_gpt_6_luna_v1: 0.631triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.631triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.631triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.631triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.631triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.631triage_gemini_4_argon_v1 · rescored 77adc97b: 0.631triage_gpt_61_sol_v1 · rescored 77adc97b: 0.631triage_gpt_6_luna_v1 · rescored 77adc97b: 0.631triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.631triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.631triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.631triage_local_dgx_nemotron_35_bolt_e4b_v1: 0.6triage_local_x3_nemotron_35_lightning_30b_v1: 0.6triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.6triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.6triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.6triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.6triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.6triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.6triage_hosted_qwen3_30b_a3b_v1: 0.585triage_kimi_k31_pro_v1: 0.585triage_local_x3_gpt_oss_20b_v1: 0.585triage_sonnet_55_v1: 0.585triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.585triage_sonnet_55_v1 · rescored 3d3a6afd: 0.585triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.585triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.585triage_sonnet_55_v1 · rescored 5f2c057c: 0.585triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.585triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.585triage_sonnet_55_v1 · rescored 77adc97b: 0.585triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.585triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.585triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0.585triage_sonnet_55_v1 · rescored cd35dc1f: 0.585triage_local_dgx_gemma4_12b_noreason_v1: 0.569triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0.569triage_local_mac_gemma4_12b_noreason_v1: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1: 0.569triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.569triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0.569triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0.569triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.569triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0.569triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0.569triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.569triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0.569triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0.569triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.569triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0.569triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0.569triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0.569triage_hosted_qwen35_122b_v2: 0.554triage_local_dgx_nemotron_35_lightning_30b_v1: 0.554triage_local_mac_qwen3_30b_a3b_v1: 0.554triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.554triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.554triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.554triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.554triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.554triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.554triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.554triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.554triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.554triage_local_dgx_gemma4_31b_v1: 0.538triage_local_dgx_nemotron_35_super_120b_v1: 0.538triage_local_mac_nemotron_35_bolt_e4b_v2: 0.538triage_local_x3_qwen3_30b_a3b_v1: 0.538triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.538triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.538triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0.538triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0.538triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.538triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.538triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0.538triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0.538triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.538triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0.538triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0.538triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.538triage_baseline_pro_v1: 0.523triage_kimi_k31_pro_v2: 0.523triage_local_dgx_gpt_oss_120b_v1: 0.523triage_local_dgx_gpt_oss_20b_v1: 0.523triage_local_dgx_qwen35_122b_v1: 0.523triage_baseline_pro_v1 · rescored 3d3a6afd: 0.523triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.523triage_baseline_pro_v1 · rescored 5f2c057c: 0.523triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.523triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0.523triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 0.523triage_baseline_pro_v1 · rescored 77adc97b: 0.523triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0.523triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 0.523triage_baseline_pro_v1 · rescored cd35dc1f: 0.523triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0.523triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0.523triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 0.523triage_gemini_35_flash_lite_v1: 0.508triage_hosted_gemma4_31b_v1: 0.508triage_hosted_gpt_oss_120b_v1: 0.508triage_hosted_qwen35_122b_v1: 0.508triage_local_dgx_gpt_oss_120b_v2: 0.508triage_local_dgx_phi4_14b_v1: 0.508triage_local_mac_gemma4_31b_v1: 0.508triage_local_mac_nemotron_35_lightning_30b_v1: 0.508triage_local_mac_phi4_14b_v1: 0.508triage_local_x3_nemotron_35_bolt_e4b_v2: 0.508triage_local_x3_phi4_14b_v1: 0.508triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.508triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.508triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0.508triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0.508triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.508triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0.508triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.508triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.508triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.508triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 0.508triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0.508triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0.508triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0.508triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0.508triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0.508triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.508triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0.508triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.508triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.508triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.508triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 0.508triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0.508triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0.508triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0.508triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0.508triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0.508triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.508triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.508triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.508triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 0.508triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0.508triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0.508triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0.508triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0.508triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0.508triage_baseline_pro_noreason_v1: 0.492triage_hosted_gpt_oss_120b_v2: 0.492triage_local_dgx_mistral_medium_35_128b_v1: 0.492triage_local_mac_gpt_oss_20b_v2: 0.492triage_local_x3_gemma4_31b_v1: 0.492triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 0.492triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.492triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 0.492triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.492triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 0.492triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.492triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 0.492triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0.492triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0.492triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 0.492triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0.492triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 0.492triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0.492triage_deepseek_41flash_v1: 0.477triage_deepseek_41flash_v2: 0.477triage_local_dgx_gemma4_31b_v2: 0.477triage_local_dgx_ministral_3_14b_v1: 0.477triage_local_dgx_nemotron_35_super_120b_v2: 0.477triage_local_dgx_qwen35_122b_v2: 0.477triage_local_dgx_qwen3_30b_a3b_v1: 0.477triage_local_mac_llama31_8b_v1: 0.477triage_local_x3_gemma4_31b_v2: 0.477triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.477triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0.477triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0.477triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0.477triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 0.477triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.477triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0.477triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0.477triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.477triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.477triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0.477triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0.477triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0.477triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 0.477triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.477triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0.477triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0.477triage_deepseek_41flash_v1 · rescored 77adc97b: 0.477triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0.477triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.477triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0.477triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.477triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0.477triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.477triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0.477triage_baseline_pro_v1 · repeats ×3: 0.467triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.467triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.467triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.467triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.467triage_hosted_gemma4_31b_v2: 0.462triage_hosted_mistral_medium_35_v1: 0.462triage_local_x3_llama31_8b_v1: 0.462triage_local_x3_qwen38_27b_v2: 0.462triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.462triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.462triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0.462triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 0.462triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.462triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.462triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0.462triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 0.462triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.462triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0.462triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.462triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0.462triage_local_dgx_llama31_8b_v1: 0.446triage_local_dgx_nemotron_35_bolt_e4b_v2: 0.446triage_local_dgx_qwen38_27b_v2: 0.446triage_local_mac_gemma4_31b_v2: 0.446triage_local_mac_gpt_oss_20b_v1: 0.446triage_local_mac_qwen38_27b_v1: 0.446triage_mistral_large_4_v1: 0.446triage_mistral_large_4_v2: 0.446triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0.446triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0.446triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 0.446triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0.446triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.446triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0.446triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0.446triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 0.446triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0.446triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 0.446triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 0.446triage_mistral_large_4_v1 · rescored 5f2c057c: 0.446triage_mistral_large_4_v2 · rescored 5f2c057c: 0.446triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0.446triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 0.446triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 0.446triage_mistral_large_4_v1 · rescored 77adc97b: 0.446triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0.446triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0.446triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 0.446triage_mistral_large_4_v1 · rescored cd35dc1f: 0.446triage_local_dgx_gpt_oss_20b_v2: 0.431triage_local_dgx_qwen38_27b_v1: 0.431triage_local_x3_qwen38_27b_v1: 0.431triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.431triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0.431triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0.431triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0.431triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0.431triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0.431triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0.431triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0.431triage_local_mac_qwen38_27b_v2: 0.415triage_local_x3_gpt_oss_20b_v2: 0.415triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 0.415triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0.415triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 0.415triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 0.415triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0.415triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 0.415triage_local_mac_ministral_3_14b_v1: 0.369triage_local_mac_nemotron_35_lightning_30b_v2: 0.369triage_local_x3_ministral_3_14b_v1: 0.369triage_local_x3_nemotron_35_lightning_30b_v2: 0.369triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.369triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0.369triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.369triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.369triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0.369triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.369triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0.369triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0.369triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0.369triage_local_dgx_llama4_scout_v1: 0.354triage_local_dgx_nemotron_35_lightning_30b_v2: 0.354triage_stub_dismiss_v1: 0.354triage_stub_escalate_v1: 0.354triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 0.354triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0.354triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0.354triage_stub_dismiss_v1 · rescored 3d3a6afd: 0.354triage_stub_escalate_v1 · rescored 3d3a6afd: 0.354triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 0.354triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0.354triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 0.354triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0.354triage_stub_dismiss_v1 · rescored 5f2c057c: 0.354triage_stub_escalate_v1 · rescored 5f2c057c: 0.354triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 0.354triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 0.354triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0.354triage_stub_dismiss_v1 · rescored 77adc97b: 0.354triage_stub_escalate_v1 · rescored 77adc97b: 0.354triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 0.354triage_stub_dismiss_v1 · rescored cd35dc1f: 0.354triage_stub_escalate_v1 · rescored cd35dc1f: 0.3540.354better · 0.677

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean model_correctitemsscored against
triage_haiku_55_v10.67765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd0.67765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c0.67765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b0.67765gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f0.67765gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v20.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v10.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats0.66265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd0.66265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.66265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats0.66265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×30.65665gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×30.64165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×30.64165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×30.64165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×30.64165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×30.64165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v10.63165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v10.63165gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v10.63165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd0.63165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c0.63165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b0.63165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f0.63165gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f0.63165gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v10.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v10.60065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.60065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.60065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.60065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.60065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v10.58565gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v10.58565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v10.58565gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v10.58565gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd0.58565gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c0.58565gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b0.58565gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f0.58565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v10.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v10.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v10.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v10.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f0.56965gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f0.56965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f0.56965gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v20.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v10.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v10.55465gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.55465gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c0.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b0.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.55465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f0.55465gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v10.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v10.53865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v20.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v10.53865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f0.53865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v10.52365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v20.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v10.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v10.52365gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd0.52365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd0.52365gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c0.52365gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b0.52365gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f0.52365gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v10.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v10.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v10.50865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v10.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v20.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v10.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v10.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v10.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v10.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v20.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v10.50865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd0.50865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c0.50865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b0.50865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f0.50865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v10.49265gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v20.49265gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v10.49265gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v20.49265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v10.49265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd0.49265gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd0.49265gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd0.49265gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd0.49265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c0.49265gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c0.49265gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c0.49265gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c0.49265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c0.49265gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b0.49265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b0.49265gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f0.49265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f0.49265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v10.47765gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v20.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v20.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v10.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v20.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v20.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v10.47765gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v10.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v20.47765gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd0.47765gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c0.47765gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b0.47765gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f0.47765gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×30.46765gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×30.46765gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×30.46765gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×30.46765gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×30.46765gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v20.46265gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v10.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v10.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v20.46265gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd0.46265gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c0.46265gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b0.46265gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f0.46265gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f0.46265gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v10.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v20.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v20.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v20.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v10.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v10.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v10.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v20.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd0.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd0.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b0.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b0.44665gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b0.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f0.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f0.44665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f0.44665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f0.44665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v20.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v10.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v10.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b0.43165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f0.43165gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f0.43165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v20.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v20.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c0.41565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b0.41565gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v10.36965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v20.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v10.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v20.36965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.36965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b0.36965gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f0.36965gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f0.36965gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v10.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v20.35465gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v10.35465gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v10.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd0.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd0.35465gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd0.35465gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd0.35465gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd0.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c0.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c0.35465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c0.35465gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c0.35465gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c0.35465gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c0.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b0.35465gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b0.35465gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b0.35465gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b0.35465gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b0.35465gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f0.35465gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f0.35465gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f0.35465gold/orrery_triage_v1_labelled gold