metric

`latency_ms`

better: lower
kind: speed: how long an item took, by the clock; needs no reference
decides: no goal
weighed by: Named-entity extraction (Few-NERD), Encyclopedia topic classification (DBpedia), Short news summaries (CNN), News topic classification (AG News), Evidence retrieval for scientific claims (SciFact), A scorer that agrees with experts on summaries (SummEval), Triage decisions on Orrery's error signals
recorded on: ag_news_200, cnn_dailymail_20, cnn_dailymail_200, cnn_dailymail_dm200, dbpedia_280, few_nerd_280, fleet_dedup_v1, orrery_triage_v1_labelled, scifact_200, smoke_v1, summeval_100

01what it measures, and how

Wall-clock time of one call, in milliseconds.

The wall-clock time of one call. When an adapter reports none, the core times the call itself; a model's first call, which loads it (the warm-up), is excluded. It depends on the machine, the provider's load and the network: compare it between arms measured under the same conditions, which the inspector's conditions line checks. Lower is better, and it needs no reference.

02on ag_news_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
ag_constant_n200_v1	0.002	200	gold/ag_news_200 gold
ag_keyword_n200_v1	0.053	200	gold/ag_news_200 gold
ag_bert_mini_n200_v1	7.110	200	gold/ag_news_200 gold
ag_bert_mini_n200_v1 · linux	20.9	200	gold/ag_news_200 gold
ag_bert_base_ta_n200_v1 · linux	56.6	200	gold/ag_news_200 gold
ag_bert_base_ta_n200_v1	76.3	200	gold/ag_news_200 gold
ag_bert_base_fy_n200_v1	90.0	200	gold/ag_news_200 gold
ag_bert_base_fy_n200_v1 · linux	201.5	200	gold/ag_news_200 gold
ag_mistral_m_n200_v1	436.6	200	gold/ag_news_200 gold
ag_gemma_s_n200_v1	664.2	200	gold/ag_news_200 gold
ag_gemma_m_n200_v1	666.2	200	gold/ag_news_200 gold
ag_mistral_s_n200_v1	731.9	200	gold/ag_news_200 gold
ag_llama_s_n200_v1	784.0	200	gold/ag_news_200 gold
ag_llama_l_n200_v1	804.5	200	gold/ag_news_200 gold
ag_openai_s_n200_v1	859.0	200	gold/ag_news_200 gold
ag_deepseek_m_n200_v1	889.7	200	gold/ag_news_200 gold
ag_qwen_m_n200_v1	910.8	200	gold/ag_news_200 gold
ag_gemma_l_n200_v1	911.1	200	gold/ag_news_200 gold
ag_anthropic_s_n200_v1	987.0	200	gold/ag_news_200 gold
ag_openai_m_n200_v1	1105.4	200	gold/ag_news_200 gold
ag_qwen_s_n200_v1	1144.1	200	gold/ag_news_200 gold
ag_glm_s_n200_v1	1153.8	200	gold/ag_news_200 gold
ag_llama_m_n200_v1	1193.4	200	gold/ag_news_200 gold
ag_openai_l_n200_v1	1207.3	200	gold/ag_news_200 gold
ag_deepseek_l_n200_v1	1323.2	200	gold/ag_news_200 gold
ag_bart_mnli_n200_v1	1386.5	200	gold/ag_news_200 gold
ag_deepseek_s_n200_v1	1602.5	200	gold/ag_news_200 gold
ag_glm_l_n200_v1	1870.9	200	gold/ag_news_200 gold
ag_anthropic_m_n200_v1	1915.9	200	gold/ag_news_200 gold
ag_anthropic_l_n200_v1	1965.1	200	gold/ag_news_200 gold
ag_glm_m_n200_v1	2261.9	200	gold/ag_news_200 gold
ag_qwen_l_n200_v1	2849.4	200	gold/ag_news_200 gold
ag_mistral_l_n200_v1	7662.1	200	gold/ag_news_200 gold
ag_constant_n200_v1 · adjudicated	0.002	200	gold/ag_news_200_adj1 gold
ag_keyword_n200_v1 · adjudicated	0.053	200	gold/ag_news_200_adj1 gold
ag_bert_mini_n200_v1 · adjudicated	7.110	200	gold/ag_news_200_adj1 gold
ag_bert_base_ta_n200_v1 · adjudicated	76.3	200	gold/ag_news_200_adj1 gold
ag_bert_base_fy_n200_v1 · adjudicated	90.0	200	gold/ag_news_200_adj1 gold
ag_mistral_m_n200_v1 · adjudicated	436.6	200	gold/ag_news_200_adj1 gold
ag_gemma_s_n200_v1 · adjudicated	664.2	200	gold/ag_news_200_adj1 gold
ag_gemma_m_n200_v1 · adjudicated	666.2	200	gold/ag_news_200_adj1 gold
ag_mistral_s_n200_v1 · adjudicated	731.9	200	gold/ag_news_200_adj1 gold
ag_llama_s_n200_v1 · adjudicated	784.0	200	gold/ag_news_200_adj1 gold
ag_llama_l_n200_v1 · adjudicated	804.5	200	gold/ag_news_200_adj1 gold
ag_openai_s_n200_v1 · adjudicated	859.0	200	gold/ag_news_200_adj1 gold
ag_deepseek_m_n200_v1 · adjudicated	889.7	200	gold/ag_news_200_adj1 gold
ag_qwen_m_n200_v1 · adjudicated	910.8	200	gold/ag_news_200_adj1 gold
ag_gemma_l_n200_v1 · adjudicated	911.1	200	gold/ag_news_200_adj1 gold
ag_anthropic_s_n200_v1 · adjudicated	987.0	200	gold/ag_news_200_adj1 gold
ag_openai_m_n200_v1 · adjudicated	1105.4	200	gold/ag_news_200_adj1 gold
ag_qwen_s_n200_v1 · adjudicated	1144.1	200	gold/ag_news_200_adj1 gold
ag_glm_s_n200_v1 · adjudicated	1153.8	200	gold/ag_news_200_adj1 gold
ag_llama_m_n200_v1 · adjudicated	1193.4	200	gold/ag_news_200_adj1 gold
ag_openai_l_n200_v1 · adjudicated	1207.3	200	gold/ag_news_200_adj1 gold
ag_deepseek_l_n200_v1 · adjudicated	1323.2	200	gold/ag_news_200_adj1 gold
ag_bart_mnli_n200_v1 · adjudicated	1386.5	200	gold/ag_news_200_adj1 gold
ag_deepseek_s_n200_v1 · adjudicated	1602.5	200	gold/ag_news_200_adj1 gold
ag_glm_l_n200_v1 · adjudicated	1870.9	200	gold/ag_news_200_adj1 gold
ag_anthropic_m_n200_v1 · adjudicated	1915.9	200	gold/ag_news_200_adj1 gold
ag_anthropic_l_n200_v1 · adjudicated	1965.1	200	gold/ag_news_200_adj1 gold
ag_glm_m_n200_v1 · adjudicated	2261.9	200	gold/ag_news_200_adj1 gold
ag_qwen_l_n200_v1 · adjudicated	2849.4	200	gold/ag_news_200_adj1 gold
ag_mistral_l_n200_v1 · adjudicated	7662.1	200	gold/ag_news_200_adj1 gold

03on cnn_dailymail_20

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
cnn_lead3_v1	0.071	20	gold/cnn_dailymail_20 gold
cnn_bart_l_v1	11861.3	20	gold/cnn_dailymail_20 gold

04on cnn_dailymail_200

Not drawn: cnn_anthropic_l_n200_v1 · silver, scored against the reference it wrote (its row is in the table).

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
cnn_lead3_n200_v1	0.065	200	gold/cnn_dailymail_200 gold
cnn_lead3_n200_v1 · linux	0.070	200	gold/cnn_dailymail_200 gold
cnn_openai_s_n200_v1	1276.6	200	gold/cnn_dailymail_200 gold
cnn_qwen_m_n200_v1	1534.0	200	gold/cnn_dailymail_200 gold
cnn_deepseek_m_n200_v1	1595.4	200	gold/cnn_dailymail_200 gold
cnn_glm_s_n200_v1	1610.4	200	gold/cnn_dailymail_200 gold
cnn_openai_m_n200_v1	1744.8	200	gold/cnn_dailymail_200 gold
cnn_openai_l_n200_v1	1774.8	200	gold/cnn_dailymail_200 gold
cnn_gemma_m_n200_v1	1872.5	200	gold/cnn_dailymail_200 gold
cnn_anthropic_s_n200_v1	2025.0	200	gold/cnn_dailymail_200 gold
cnn_llama_s_n200_v1	2388.0	200	gold/cnn_dailymail_200 gold
cnn_gemma_l_n200_v1	2591.1	200	gold/cnn_dailymail_200 gold
cnn_mistral_m_n200_v1	2653.7	200	gold/cnn_dailymail_200 gold
cnn_deepseek_l_n200_v1	2665.5	200	gold/cnn_dailymail_200 gold
cnn_deepseek_s_n200_v1	2752.6	200	gold/cnn_dailymail_200 gold
cnn_mistral_s_n200_v1	2819.9	200	gold/cnn_dailymail_200 gold
cnn_qwen_l_n200_v1	2944.9	200	gold/cnn_dailymail_200 gold
cnn_anthropic_m_n200_v1	3034.2	200	gold/cnn_dailymail_200 gold
cnn_glm_m_n200_v1	3230.1	200	gold/cnn_dailymail_200 gold
cnn_anthropic_l_n200_v1	3402.0	200	gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · linux	3420.4	200	gold/cnn_dailymail_200 gold
cnn_llama_l_n200_v1	3473.0	200	gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · repeats	3765.5	200	gold/cnn_dailymail_200 gold
cnn_llama_m_n200_v1	4045.2	200	gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1 · linux	4248.2	200	gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1 · linux	4374.7	200	gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1	4484.2	200	gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1	4620.6	200	gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1	5061.7	200	gold/cnn_dailymail_200 gold
cnn_glm_l_n200_v1	5689.4	200	gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1 · linux	6507.4	200	gold/cnn_dailymail_200 gold
cnn_gemma_s_n200_v1	6956.2	200	gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1	11009.6	200	gold/cnn_dailymail_200 gold
cnn_mistral_l_n200_v1	11044.7	200	gold/cnn_dailymail_200 gold
cnn_qwen_s_n200_v1	25037.7	200	gold/cnn_dailymail_200 gold
cnn_lead3_n200_v1 · silver	0.065	200	silver/cnn_dailymail_200 silver
cnn_openai_s_n200_v1 · silver	1276.6	200	silver/cnn_dailymail_200 silver
cnn_qwen_m_n200_v1 · silver	1534.0	200	silver/cnn_dailymail_200 silver
cnn_deepseek_m_n200_v1 · silver	1595.4	200	silver/cnn_dailymail_200 silver
cnn_glm_s_n200_v1 · silver	1610.4	200	silver/cnn_dailymail_200 silver
cnn_openai_m_n200_v1 · silver	1744.8	200	silver/cnn_dailymail_200 silver
cnn_openai_l_n200_v1 · silver	1774.8	200	silver/cnn_dailymail_200 silver
cnn_gemma_m_n200_v1 · silver	1872.5	200	silver/cnn_dailymail_200 silver
cnn_anthropic_s_n200_v1 · silver	2025.0	200	silver/cnn_dailymail_200 silver
cnn_llama_s_n200_v1 · silver	2388.0	200	silver/cnn_dailymail_200 silver
cnn_gemma_l_n200_v1 · silver	2591.1	200	silver/cnn_dailymail_200 silver
cnn_mistral_m_n200_v1 · silver	2653.7	200	silver/cnn_dailymail_200 silver
cnn_deepseek_l_n200_v1 · silver	2665.5	200	silver/cnn_dailymail_200 silver
cnn_deepseek_s_n200_v1 · silver	2752.6	200	silver/cnn_dailymail_200 silver
cnn_mistral_s_n200_v1 · silver	2819.9	200	silver/cnn_dailymail_200 silver
cnn_qwen_l_n200_v1 · silver	2944.9	200	silver/cnn_dailymail_200 silver
cnn_anthropic_m_n200_v1 · silver	3034.2	200	silver/cnn_dailymail_200 silver
cnn_glm_m_n200_v1 · silver	3230.1	200	silver/cnn_dailymail_200 silver
cnn_anthropic_l_n200_v1 · silver	3402.0	200	silver/cnn_dailymail_200 silver
cnn_llama_l_n200_v1 · silver	3473.0	200	silver/cnn_dailymail_200 silver
cnn_llama_m_n200_v1 · silver	4045.2	200	silver/cnn_dailymail_200 silver
cnn_bart_s_n200_v1 · silver	4484.2	200	silver/cnn_dailymail_200 silver
cnn_bart_l_xsum_n200_v1 · silver	4620.6	200	silver/cnn_dailymail_200 silver
cnn_bart_m_n200_v1 · silver	5061.7	200	silver/cnn_dailymail_200 silver
cnn_glm_l_n200_v1 · silver	5689.4	200	silver/cnn_dailymail_200 silver
cnn_gemma_s_n200_v1 · silver	6956.2	200	silver/cnn_dailymail_200 silver
cnn_bart_l_n200_v1 · silver	11009.6	200	silver/cnn_dailymail_200 silver
cnn_mistral_l_n200_v1 · silver	11044.7	200	silver/cnn_dailymail_200 silver
cnn_qwen_s_n200_v1 · silver	25037.7	200	silver/cnn_dailymail_200 silver
cnn_lead3_n200_v1 · judged/judge-summ-pick-v1	0.065	198	judge `judge-summ-pick-v1`
cnn_lead3_n200_v1 · judged/judge-summ-v1	0.065	197	judge `judge-summ-v1`
cnn_lead3_n200_v1 · judged/judge-summ-v2	0.065	200	judge `judge-summ-v2`
cnn_openai_s_n200_v1 · judged/judge-summ-v2	1276.6	200	judge `judge-summ-v2`
cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1	1276.8	198	judge `judge-summ-pick-v1`
cnn_openai_s_n200_v1 · judged/judge-summ-v1	1276.8	197	judge `judge-summ-v1`
cnn_qwen_m_n200_v1 · judged/judge-summ-v1	1531.7	197	judge `judge-summ-v1`
cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1	1533.2	198	judge `judge-summ-pick-v1`
cnn_qwen_m_n200_v1 · judged/judge-summ-v2	1534.0	200	judge `judge-summ-v2`
cnn_deepseek_m_n200_v1 · judged/judge-summ-v1	1588.5	197	judge `judge-summ-v1`
cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1	1592.0	198	judge `judge-summ-pick-v1`
cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1	1595.4	200	judge `judge-summ-pointwise-v1`
cnn_deepseek_m_n200_v1 · judged/judge-summ-v2	1595.4	200	judge `judge-summ-v2`
cnn_glm_s_n200_v1 · judged/judge-summ-v2	1610.4	200	judge `judge-summ-v2`
cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1	1611.8	198	judge `judge-summ-pick-v1`
cnn_glm_s_n200_v1 · judged/judge-summ-v1	1615.3	197	judge `judge-summ-v1`
cnn_openai_m_n200_v1 · judged/judge-summ-v2	1744.8	200	judge `judge-summ-v2`
cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1	1745.7	198	judge `judge-summ-pick-v1`
cnn_openai_m_n200_v1 · judged/judge-summ-v1	1746.2	197	judge `judge-summ-v1`
cnn_openai_l_n200_v1 · judged/judge-summ-v1	1769.2	197	judge `judge-summ-v1`
cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1	1774.7	198	judge `judge-summ-pick-v1`
cnn_openai_l_n200_v1 · judged/judge-summ-v2	1774.8	200	judge `judge-summ-v2`
cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1	1871.0	198	judge `judge-summ-pick-v1`
cnn_gemma_m_n200_v1 · judged/judge-summ-v2	1872.5	200	judge `judge-summ-v2`
cnn_gemma_m_n200_v1 · judged/judge-summ-v1	1873.0	197	judge `judge-summ-v1`
cnn_anthropic_s_n200_v1 · judged/judge-summ-v1	2019.8	197	judge `judge-summ-v1`
cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1	2022.6	198	judge `judge-summ-pick-v1`
cnn_anthropic_s_n200_v1 · judged/judge-summ-v2	2025.0	200	judge `judge-summ-v2`
cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1	2369.9	198	judge `judge-summ-pick-v1`
cnn_llama_s_n200_v1 · judged/judge-summ-v2	2388.0	200	judge `judge-summ-v2`
cnn_llama_s_n200_v1 · judged/judge-summ-v1	2391.7	197	judge `judge-summ-v1`
cnn_gemma_l_n200_v1 · judged/judge-summ-v2	2591.1	200	judge `judge-summ-v2`
cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1	2600.4	198	judge `judge-summ-pick-v1`
cnn_gemma_l_n200_v1 · judged/judge-summ-v1	2601.4	197	judge `judge-summ-v1`
cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1	2648.0	198	judge `judge-summ-pick-v1`
cnn_mistral_m_n200_v1 · judged/judge-summ-v2	2653.7	200	judge `judge-summ-v2`
cnn_mistral_m_n200_v1 · judged/judge-summ-v1	2658.6	197	judge `judge-summ-v1`
cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1	2658.8	198	judge `judge-summ-pick-v1`
cnn_deepseek_l_n200_v1 · judged/judge-summ-v2	2665.5	200	judge `judge-summ-v2`
cnn_deepseek_l_n200_v1 · judged/judge-summ-v1	2665.9	197	judge `judge-summ-v1`
cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1	2713.6	198	judge `judge-summ-pick-v1`
cnn_deepseek_s_n200_v1 · judged/judge-summ-v2	2752.6	200	judge `judge-summ-v2`
cnn_deepseek_s_n200_v1 · judged/judge-summ-v1	2752.8	197	judge `judge-summ-v1`
cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1	2810.1	198	judge `judge-summ-pick-v1`
cnn_mistral_s_n200_v1 · judged/judge-summ-v2	2819.9	200	judge `judge-summ-v2`
cnn_mistral_s_n200_v1 · judged/judge-summ-v1	2826.3	197	judge `judge-summ-v1`
cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1	2943.1	198	judge `judge-summ-pick-v1`
cnn_qwen_l_n200_v1 · judged/judge-summ-v2	2944.9	200	judge `judge-summ-v2`
cnn_qwen_l_n200_v1 · judged/judge-summ-v1	2946.3	197	judge `judge-summ-v1`
cnn_anthropic_m_n200_v1 · judged/judge-summ-v1	3031.5	197	judge `judge-summ-v1`
cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1	3032.8	198	judge `judge-summ-pick-v1`
cnn_anthropic_m_n200_v1 · judged/judge-summ-v2	3034.2	200	judge `judge-summ-v2`
cnn_glm_m_n200_v1 · judged/judge-summ-v2	3230.1	200	judge `judge-summ-v2`
cnn_glm_m_n200_v1 · judged/judge-summ-v1	3234.1	197	judge `judge-summ-v1`
cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1	3243.3	198	judge `judge-summ-pick-v1`
cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1	3398.7	198	judge `judge-summ-pick-v1`
cnn_anthropic_l_n200_v1 · judged/judge-summ-v2	3402.0	200	judge `judge-summ-v2`
cnn_anthropic_l_n200_v1 · judged/judge-summ-v1	3402.2	197	judge `judge-summ-v1`
cnn_llama_l_n200_v1 · judged/judge-summ-v2	3473.0	200	judge `judge-summ-v2`
cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1	3474.2	198	judge `judge-summ-pick-v1`
cnn_llama_l_n200_v1 · judged/judge-summ-v1	3495.5	197	judge `judge-summ-v1`
cnn_llama_m_n200_v1 · judged/judge-summ-v2	4045.2	200	judge `judge-summ-v2`
cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1	4066.1	198	judge `judge-summ-pick-v1`
cnn_llama_m_n200_v1 · judged/judge-summ-v1	4072.3	197	judge `judge-summ-v1`
cnn_bart_s_n200_v1 · judged/judge-summ-v2	4484.2	200	judge `judge-summ-v2`
cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1	4484.8	198	judge `judge-summ-pick-v1`
cnn_bart_s_n200_v1 · judged/judge-summ-v1	4488.9	197	judge `judge-summ-v1`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1	4604.6	197	judge `judge-summ-v1`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1	4609.2	198	judge `judge-summ-pick-v1`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2	4620.6	200	judge `judge-summ-v2`
cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1	5047.9	198	judge `judge-summ-pick-v1`
cnn_bart_m_n200_v1 · judged/judge-summ-v1	5054.3	197	judge `judge-summ-v1`
cnn_bart_m_n200_v1 · judged/judge-summ-v2	5061.7	200	judge `judge-summ-v2`
cnn_glm_l_n200_v1 · judged/judge-summ-v2	5689.4	200	judge `judge-summ-v2`
cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1	5718.2	198	judge `judge-summ-pick-v1`
cnn_glm_l_n200_v1 · judged/judge-summ-v1	5738.7	197	judge `judge-summ-v1`
cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1	6932.0	198	judge `judge-summ-pick-v1`
cnn_gemma_s_n200_v1 · judged/judge-summ-v2	6956.2	200	judge `judge-summ-v2`
cnn_gemma_s_n200_v1 · judged/judge-summ-v1	6988.6	197	judge `judge-summ-v1`
cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1	10528.6	198	judge `judge-summ-pick-v1`
cnn_bart_l_n200_v1 · judged/judge-summ-v1	10995.7	197	judge `judge-summ-v1`
cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1	11006.8	198	judge `judge-summ-pick-v1`
cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1	11009.6	200	judge `judge-summ-pointwise-v1`
cnn_bart_l_n200_v1 · judged/judge-summ-v2	11009.6	200	judge `judge-summ-v2`
cnn_mistral_l_n200_v1 · judged/judge-summ-v2	11044.7	200	judge `judge-summ-v2`
cnn_mistral_l_n200_v1 · judged/judge-summ-v1	11179.5	197	judge `judge-summ-v1`
cnn_qwen_s_n200_v1 · judged/judge-summ-v1	24871.6	197	judge `judge-summ-v1`
cnn_qwen_s_n200_v1 · judged/judge-summ-v2	25037.7	200	judge `judge-summ-v2`
cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1	25217.6	198	judge `judge-summ-pick-v1`

05on cnn_dailymail_dm200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
cnn_lead3_dm200_v1	0.123	200	gold/cnn_dailymail_dm200 gold
cnn_bart_s_dm200_v1	4302.1	200	gold/cnn_dailymail_dm200 gold
cnn_bart_l_xsum_dm200_v1	5362.0	200	gold/cnn_dailymail_dm200 gold
cnn_bart_m_dm200_v1	7292.4	200	gold/cnn_dailymail_dm200 gold
cnn_bart_l_dm200_v1	11259.5	200	gold/cnn_dailymail_dm200 gold

06on dbpedia_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
db_constant_n200_v1	0.003	280	gold/dbpedia_280 gold
db_keyword_n200_v1	0.100	280	gold/dbpedia_280 gold
db_bert_base_fy_n200_v1	86.7	280	gold/dbpedia_280 gold
db_bert_base_fy_n200_v1 · linux	165.8	280	gold/dbpedia_280 gold
db_mistral_m_n200_v1	439.5	280	gold/dbpedia_280 gold
db_gemma_s_n200_v1	647.2	280	gold/dbpedia_280 gold
db_llama_l_n200_v1	659.8	280	gold/dbpedia_280 gold
db_qwen_m_n200_v1	853.2	280	gold/dbpedia_280 gold
db_gemma_m_n200_v1	856.5	280	gold/dbpedia_280 gold
db_openai_s_n200_v1	909.8	280	gold/dbpedia_280 gold
db_mistral_s_n200_v1	942.7	280	gold/dbpedia_280 gold
db_deepseek_m_n200_v1	974.6	280	gold/dbpedia_280 gold
db_anthropic_s_n200_v1	1082.6	280	gold/dbpedia_280 gold
db_gemma_l_n200_v1	1145.4	280	gold/dbpedia_280 gold
db_openai_m_n200_v1	1178.0	280	gold/dbpedia_280 gold
db_glm_s_n200_v1	1183.0	280	gold/dbpedia_280 gold
db_openai_l_n200_v1	1266.3	280	gold/dbpedia_280 gold
db_llama_s_n200_v1	1374.5	280	gold/dbpedia_280 gold
db_deepseek_s_n200_v1	1548.3	280	gold/dbpedia_280 gold
db_qwen_l_n200_v1	1660.0	280	gold/dbpedia_280 gold
db_deepseek_l_n200_v1	1781.5	280	gold/dbpedia_280 gold
db_llama_m_n200_v1	1927.0	280	gold/dbpedia_280 gold
db_glm_m_n200_v1	1930.1	280	gold/dbpedia_280 gold
db_glm_l_n200_v1	1978.0	280	gold/dbpedia_280 gold
db_anthropic_m_n200_v1	2257.9	280	gold/dbpedia_280 gold
db_anthropic_l_n200_v1	3101.1	280	gold/dbpedia_280 gold
db_bart_mnli_n200_v1	5717.7	280	gold/dbpedia_280 gold
db_mistral_l_n200_v1	10565.4	280	gold/dbpedia_280 gold
db_qwen_s_n200_v1	12349.1	280	gold/dbpedia_280 gold

07on few_nerd_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
fn_nothing_n200_v1	0.002	280	gold/few_nerd_280 gold
fn_capitalized_n200_v1	0.035	280	gold/few_nerd_280 gold
fn_gliner_n200_v1	169.4	280	gold/few_nerd_280 gold
fn_span_marker_n200_v1	757.9	280	gold/few_nerd_280 gold
fn_mistral_m_n200_v1	793.7	280	gold/few_nerd_280 gold
fn_openai_s_n200_v1	1205.6	280	gold/few_nerd_280 gold
fn_deepseek_m_n200_v1	1228.7	280	gold/few_nerd_280 gold
fn_anthropic_s_n200_v1	1337.5	280	gold/few_nerd_280 gold
fn_openai_m_n200_v1	1598.1	280	gold/few_nerd_280 gold
fn_qwen_l_n200_v1	1667.2	280	gold/few_nerd_280 gold
fn_openai_l_n200_v1	1903.2	280	gold/few_nerd_280 gold
fn_mistral_s_n200_v1	2063.6	280	gold/few_nerd_280 gold
fn_glm_s_n200_v1	2110.7	280	gold/few_nerd_280 gold
fn_qwen_s_n200_v1	2153.7	280	gold/few_nerd_280 gold
fn_qwen_m_n200_v1	2241.4	280	gold/few_nerd_280 gold
fn_deepseek_s_n200_v1	2277.1	280	gold/few_nerd_280 gold
fn_llama_s_n200_v1	2637.6	280	gold/few_nerd_280 gold
fn_anthropic_m_n200_v1	2691.7	280	gold/few_nerd_280 gold
fn_anthropic_l_n200_v1	2759.0	280	gold/few_nerd_280 gold
fn_gemma_l_n200_v1	2910.5	280	gold/few_nerd_280 gold
fn_glm_l_n200_v1	3013.7	280	gold/few_nerd_280 gold
fn_gemma_m_n200_v1	3049.6	280	gold/few_nerd_280 gold
fn_glm_m_n200_v1	3658.9	280	gold/few_nerd_280 gold
fn_deepseek_l_n200_v1	3726.8	280	gold/few_nerd_280 gold
fn_llama_l_n200_v1	4796.9	280	gold/few_nerd_280 gold
fn_llama_m_n200_v1	5402.1	280	gold/few_nerd_280 gold
fn_gemma_s_n200_v1	5941.2	280	gold/few_nerd_280 gold
fn_mistral_l_n200_v1	10247.1	280	gold/few_nerd_280 gold

08on fleet_dedup_v1

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
dedup_guard_v1	0.136	100	silver/fleet_dedup_v1 silver
dedup_pinned_f07efa9_v1	0.174	100	—

09on orrery_triage_v1_labelled

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
triage_stub_escalate_v1	0.090	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd	0.090	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c	0.090	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b	0.090	65	gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f	0.090	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1	0.103	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd	0.103	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c	0.103	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b	0.103	65	gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f	0.103	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2	3281.9	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd	3281.9	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c	3281.9	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1	3487.3	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd	3487.3	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c	3487.3	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b	3487.3	65	gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f	3487.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2	3880.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd	3880.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c	3880.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2	4680.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd	4680.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c	4680.7	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2	5646.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd	5646.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c	5646.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1	5697.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd	5697.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c	5697.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b	5697.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f	5697.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1	5766.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd	5766.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c	5766.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b	5766.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f	5766.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2	5802.5	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd	5802.5	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c	5802.5	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1	5836.8	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd	5836.8	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c	5836.8	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b	5836.8	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f	5836.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1	5859.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd	5859.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c	5859.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b	5859.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f	5859.1	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2	5863.5	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd	5863.5	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c	5863.5	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2	5883.1	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd	5883.1	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c	5883.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1	6103.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd	6103.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c	6103.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b	6103.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f	6103.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2	6540.7	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd	6540.7	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c	6540.7	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2	6681.4	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd	6681.4	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c	6681.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2	6785.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd	6785.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c	6785.7	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1	6932.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd	6932.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c	6932.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b	6932.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f	6932.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2	7391.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd	7391.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c	7391.1	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×3	7528.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3	7528.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3	7528.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×3	7528.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3	7528.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1	7625.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd	7625.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c	7625.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b	7625.9	65	gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f	7625.9	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1	7780.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd	7780.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c	7780.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b	7780.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f	7780.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2	7797.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd	7797.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c	7797.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1	7824.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd	7824.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c	7824.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b	7824.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f	7824.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2	7973.3	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd	7973.3	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c	7973.3	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1	8352.2	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd	8352.2	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c	8352.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1	8996.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd	8996.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c	8996.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b	8996.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f	8996.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1	9174.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd	9174.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c	9174.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b	9174.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f	9174.2	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1	9832.8	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c	9832.8	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b	9832.8	65	gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f	9832.8	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1	10510.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c	10510.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b	10510.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f	10510.3	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2	10549.7	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd	10549.7	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c	10549.7	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2	11064.9	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd	11064.9	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c	11064.9	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×3	11597.7	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3	11597.7	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3	11597.7	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3	11597.7	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×3	12005.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3	12005.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3	12005.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3	12005.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3	12005.5	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1	12035.0	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd	12035.0	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c	12035.0	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b	12035.0	65	gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f	12035.0	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1	12200.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c	12200.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b	12200.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f	12200.5	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1	12241.4	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd	12241.4	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c	12241.4	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b	12241.4	65	gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f	12241.4	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2	13030.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd	13030.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c	13030.5	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1	13354.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd	13354.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c	13354.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b	13354.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f	13354.0	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1	13455.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c	13455.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b	13455.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f	13455.9	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2	14320.1	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd	14320.1	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c	14320.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2	14511.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd	14511.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c	14511.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1	14765.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c	14765.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b	14765.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f	14765.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1	15258.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c	15258.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b	15258.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f	15258.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2	15897.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd	15897.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c	15897.7	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2	16558.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd	16558.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c	16558.9	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1	16911.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd	16911.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c	16911.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b	16911.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f	16911.2	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1	16973.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd	16973.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c	16973.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b	16973.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f	16973.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1	17021.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c	17021.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b	17021.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f	17021.1	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1	17235.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c	17235.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b	17235.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f	17235.0	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1	17290.2	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c	17290.2	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b	17290.2	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f	17290.2	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1	17661.6	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c	17661.6	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b	17661.6	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f	17661.6	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1	17976.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd	17976.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c	17976.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b	17976.1	65	gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f	17976.1	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1	19952.6	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c	19952.6	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b	19952.6	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f	19952.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1	20452.2	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd	20452.2	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c	20452.2	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b	20452.2	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f	20452.2	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2	20489.0	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd	20489.0	65	gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c	20489.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2	20630.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd	20630.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c	20630.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats	21266.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats	21266.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats	21266.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats	21266.2	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2	21532.6	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd	21532.6	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c	21532.6	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1	21773.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd	21773.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c	21773.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b	21773.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f	21773.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1	22057.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd	22057.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c	22057.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b	22057.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f	22057.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1	22252.4	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c	22252.4	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b	22252.4	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f	22252.4	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1	22736.3	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd	22736.3	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c	22736.3	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b	22736.3	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f	22736.3	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1	23126.4	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c	23126.4	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b	23126.4	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f	23126.4	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2	25063.1	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd	25063.1	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c	25063.1	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×3	26284.7	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3	26284.7	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3	26284.7	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3	26284.7	65	gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3	26284.7	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1	27377.7	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd	27377.7	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c	27377.7	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b	27377.7	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f	27377.7	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1	30118.2	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd	30118.2	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c	30118.2	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b	30118.2	65	gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f	30118.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2	33394.6	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd	33394.6	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c	33394.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1	37825.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c	37825.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b	37825.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f	37825.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1	38007.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c	38007.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b	38007.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f	38007.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1	39458.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c	39458.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b	39458.3	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f	39458.3	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1	39979.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c	39979.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b	39979.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f	39979.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1	41018.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c	41018.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b	41018.6	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f	41018.6	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1	46089.2	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c	46089.2	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b	46089.2	65	gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f	46089.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1	46161.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c	46161.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b	46161.2	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f	46161.2	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1	58862.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c	58862.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b	58862.8	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f	58862.8	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1	66004.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c	66004.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b	66004.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f	66004.2	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1	69674.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c	69674.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b	69674.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f	69674.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1	72058.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c	72058.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b	72058.0	65	gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f	72058.0	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1	100888.9	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c	100888.9	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b	100888.9	65	gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f	100888.9	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1	117538.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd	117538.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c	117538.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b	117538.1	65	gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f	117538.1	65	gold/orrery_triage_v1_labelled gold

10on scifact_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
sf_first_k_n200_v1	0.253	200	gold/scifact_200 gold
sf_random_n200_v1	1.631	200	gold/scifact_200 gold
sf_minilm_n200_v1	17.2	200	gold/scifact_200 gold
sf_bm25_n200_v1	22.3	200	gold/scifact_200 gold
sf_bge_small_n200_v1	29.4	200	gold/scifact_200 gold
sf_e5_base_n200_v1	59.4	200	gold/scifact_200 gold
sf_mpnet_n200_v1	65.4	200	gold/scifact_200 gold
sf_deepseek_m_n200_v1	1145.0	200	gold/scifact_200 gold
sf_glm_s_bm25first_n200_v1 · pair	3023.9	200	gold/scifact_200 gold
sf_glm_s_n200_v1	3119.5	200	gold/scifact_200 gold
sf_glm_s_e5first_n200_v1 · pair	3126.9	200	gold/scifact_200 gold
sf_deepseek_s_n200_v1	3452.8	200	gold/scifact_200 gold
sf_llama_m_n200_v1	4064.0	200	gold/scifact_200 gold
sf_llama_l_n200_v1	4553.7	200	gold/scifact_200 gold
sf_llama_s_n200_v1 · repeats	4873.8	200	gold/scifact_200 gold
sf_qwen_s_n200_v1	5650.6	200	gold/scifact_200 gold
sf_gemma_m_n200_v1	5663.7	200	gold/scifact_200 gold
sf_qwen_s_n200_v1 · repeats	5976.0	200	gold/scifact_200 gold
sf_qwen_s_e5first_n200_v1 · pair	6145.4	200	gold/scifact_200 gold
sf_qwen_s_bm25first_n200_v1 · pair	6196.2	200	gold/scifact_200 gold
sf_glm_m_n200_v1	6920.1	200	gold/scifact_200 gold
sf_gemma_s_n200_v1	7071.1	200	gold/scifact_200 gold
sf_mistral_s_n200_v1	7223.8	200	gold/scifact_200 gold
sf_gemma_l_n200_v1	9701.0	200	gold/scifact_200 gold
sf_llama_l_n200_v1 · repeats	11385.9	200	gold/scifact_200 gold
sf_llama_s_n200_v1	17481.6	200	gold/scifact_200 gold

11on smoke_v1

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
demo_v1	0.009	5	silver/smoke_v1 silver

12on summeval_100

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean latency_ms	items	scored against
se_random_n100_pick_v1	0.044	100	gold/summeval_100 gold
se_random_n100_pairwise_v1	0.073	100	gold/summeval_100 gold
se_random_n100_v1	0.131	100	gold/summeval_100 gold
se_length_n100_v1	0.192	100	gold/summeval_100 gold
se_rouge2_n100_v1	180.9	100	gold/summeval_100 gold
se_rouge1_n100_v1	195.6	100	gold/summeval_100 gold
se_rougeL_n100_v1	274.3	100	gold/summeval_100 gold
se_coverage_n100_v1	892.0	100	gold/summeval_100 gold
se_deepseek_m_n100_v1	1120.3	100	gold/summeval_100 gold
se_qwen_s_n100_pick_v1	1492.1	100	gold/summeval_100 gold
se_openai_s_n100_v1	1550.9	100	gold/summeval_100 gold
se_gemma_l_n100_pick_v1	1813.0	100	gold/summeval_100 gold
se_mistral_m_n100_v1	1945.6	100	gold/summeval_100 gold
se_openai_l_n100_v1	2227.9	100	gold/summeval_100 gold
se_anthropic_s_n100_v1	2432.6	100	gold/summeval_100 gold
se_gemma_l_pubtext_n100_v1 · pair	2614.2	100	gold/summeval_100 gold
se_openai_m_n100_v1	2689.5	100	gold/summeval_100 gold
se_anthropic_m_n100_v1	3008.2	100	gold/summeval_100 gold
se_gemma_l_pubtextcw_n100_v1 · pair	3268.1	100	gold/summeval_100 gold
se_anthropic_l_n100_v1	3426.6	100	gold/summeval_100 gold
se_glm_l_n100_v1	3589.8	100	gold/summeval_100 gold
se_glm_m_n100_v1	3608.7	100	gold/summeval_100 gold
se_qwen_s_n100_v1	3953.6	100	gold/summeval_100 gold
se_deepseek_s_n100_v1	3984.8	100	gold/summeval_100 gold
se_gemma_m_n100_v1	4308.1	100	gold/summeval_100 gold
se_qwen37max_n100_v1	4514.3	100	gold/summeval_100 gold
se_llama_m_n100_v1	4834.2	100	gold/summeval_100 gold
se_deepseek_l_n100_v1	5535.2	100	gold/summeval_100 gold
se_glm_s_n100_v1	5617.5	100	gold/summeval_100 gold
se_gemma_l_n100_v1	6670.3	100	gold/summeval_100 gold
se_qwen_m_n100_v1	6770.4	100	gold/summeval_100 gold
se_gemma_l_n100_pairwise_v1	6980.8	100	gold/summeval_100 gold
se_llama_s_n100_v1	7141.3	100	gold/summeval_100 gold
se_mistral_s_n100_v1	9184.9	100	gold/summeval_100 gold
se_mistral_l_n100_v1	10884.9	100	gold/summeval_100 gold
se_gemma_s_n100_v1	11572.0	100	gold/summeval_100 gold
se_llama_l_n100_v1	12345.9	100	gold/summeval_100 gold
se_bartscore_n100_v1	15662.3	100	gold/summeval_100 gold
se_qwen_s_n100_pairwise_v1	20081.2	100	gold/summeval_100 gold
se_unieval_n100_v1	41676.1	100	gold/summeval_100 gold
se_unievalpub_n100_v1	44694.4	100	gold/summeval_100 gold