eval-harness

metric

tokens_in

better
lower
kind
cost: what an item cost, from the bill or the tokens; needs no reference
decides
no goal
weighed by
no goal
recorded on
ag_news_200, cnn_dailymail_20, cnn_dailymail_200, cnn_dailymail_dm200, dbpedia_280, few_nerd_280, orrery_triage_v1_labelled, scifact_200, summeval_100

01what it measures, and how

Prompt tokens per item, as the provider counted them.

Counted by the provider, not the harness: an adapter that reports none leaves it empty. Together with the config's prices it is the fallback cost when no bill comes back.

02on ag_news_200

gold/ag_news_200ag_deepseek_l_n200_v1: 93.9ag_deepseek_m_n200_v1: 93.9ag_deepseek_s_n200_v1: 93.9ag_openai_l_n200_v1: 94.8ag_openai_m_n200_v1: 94.8ag_openai_s_n200_v1: 94.8ag_glm_l_n200_v1: 95.4ag_mistral_l_n200_v1: 95.5ag_mistral_s_n200_v1: 95.5ag_glm_m_n200_v1: 96.8ag_llama_l_n200_v1: 98.1ag_llama_s_n200_v1: 98.1ag_glm_s_n200_v1: 98.3ag_qwen_l_n200_v1: 98.4ag_gemma_s_n200_v1: 100.0ag_qwen_m_n200_v1: 103.0ag_llama_m_n200_v1: 104.0ag_gemma_l_n200_v1: 104.0ag_gemma_m_n200_v1: 104.0ag_anthropic_s_n200_v1: 104.8ag_mistral_m_n200_v1: 107.5ag_qwen_s_n200_v1: 116.0ag_anthropic_l_n200_v1: 149.8ag_anthropic_m_n200_v1: 150.0gold/ag_news_200_adj1ag_deepseek_l_n200_v1 · adjudicated: 93.9ag_deepseek_m_n200_v1 · adjudicated: 93.9ag_deepseek_s_n200_v1 · adjudicated: 93.9ag_openai_l_n200_v1 · adjudicated: 94.8ag_openai_m_n200_v1 · adjudicated: 94.8ag_openai_s_n200_v1 · adjudicated: 94.8ag_glm_l_n200_v1 · adjudicated: 95.4ag_mistral_l_n200_v1 · adjudicated: 95.5ag_mistral_s_n200_v1 · adjudicated: 95.5ag_glm_m_n200_v1 · adjudicated: 96.8ag_llama_l_n200_v1 · adjudicated: 98.1ag_llama_s_n200_v1 · adjudicated: 98.1ag_glm_s_n200_v1 · adjudicated: 98.3ag_qwen_l_n200_v1 · adjudicated: 98.4ag_gemma_s_n200_v1 · adjudicated: 100.0ag_qwen_m_n200_v1 · adjudicated: 103.0ag_llama_m_n200_v1 · adjudicated: 104.0ag_gemma_l_n200_v1 · adjudicated: 104.0ag_gemma_m_n200_v1 · adjudicated: 104.0ag_anthropic_s_n200_v1 · adjudicated: 104.8ag_mistral_m_n200_v1 · adjudicated: 107.5ag_qwen_s_n200_v1 · adjudicated: 116.0ag_anthropic_l_n200_v1 · adjudicated: 149.8ag_anthropic_m_n200_v1 · adjudicated: 150.093.9 · better150.0

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
ag_deepseek_l_n200_v193.9200gold/ag_news_200 gold
ag_deepseek_m_n200_v193.9200gold/ag_news_200 gold
ag_deepseek_s_n200_v193.9200gold/ag_news_200 gold
ag_openai_l_n200_v194.8200gold/ag_news_200 gold
ag_openai_m_n200_v194.8200gold/ag_news_200 gold
ag_openai_s_n200_v194.8200gold/ag_news_200 gold
ag_glm_l_n200_v195.4200gold/ag_news_200 gold
ag_mistral_l_n200_v195.5200gold/ag_news_200 gold
ag_mistral_s_n200_v195.5200gold/ag_news_200 gold
ag_glm_m_n200_v196.8200gold/ag_news_200 gold
ag_llama_l_n200_v198.1200gold/ag_news_200 gold
ag_llama_s_n200_v198.1200gold/ag_news_200 gold
ag_glm_s_n200_v198.3200gold/ag_news_200 gold
ag_qwen_l_n200_v198.4200gold/ag_news_200 gold
ag_gemma_s_n200_v1100.0200gold/ag_news_200 gold
ag_qwen_m_n200_v1103.0200gold/ag_news_200 gold
ag_llama_m_n200_v1104.0200gold/ag_news_200 gold
ag_gemma_l_n200_v1104.0200gold/ag_news_200 gold
ag_gemma_m_n200_v1104.0200gold/ag_news_200 gold
ag_anthropic_s_n200_v1104.8200gold/ag_news_200 gold
ag_mistral_m_n200_v1107.5200gold/ag_news_200 gold
ag_qwen_s_n200_v1116.0200gold/ag_news_200 gold
ag_anthropic_l_n200_v1149.8200gold/ag_news_200 gold
ag_anthropic_m_n200_v1150.0200gold/ag_news_200 gold
ag_deepseek_l_n200_v1 · adjudicated93.9200gold/ag_news_200_adj1 gold
ag_deepseek_m_n200_v1 · adjudicated93.9200gold/ag_news_200_adj1 gold
ag_deepseek_s_n200_v1 · adjudicated93.9200gold/ag_news_200_adj1 gold
ag_openai_l_n200_v1 · adjudicated94.8200gold/ag_news_200_adj1 gold
ag_openai_m_n200_v1 · adjudicated94.8200gold/ag_news_200_adj1 gold
ag_openai_s_n200_v1 · adjudicated94.8200gold/ag_news_200_adj1 gold
ag_glm_l_n200_v1 · adjudicated95.4200gold/ag_news_200_adj1 gold
ag_mistral_l_n200_v1 · adjudicated95.5200gold/ag_news_200_adj1 gold
ag_mistral_s_n200_v1 · adjudicated95.5200gold/ag_news_200_adj1 gold
ag_glm_m_n200_v1 · adjudicated96.8200gold/ag_news_200_adj1 gold
ag_llama_l_n200_v1 · adjudicated98.1200gold/ag_news_200_adj1 gold
ag_llama_s_n200_v1 · adjudicated98.1200gold/ag_news_200_adj1 gold
ag_glm_s_n200_v1 · adjudicated98.3200gold/ag_news_200_adj1 gold
ag_qwen_l_n200_v1 · adjudicated98.4200gold/ag_news_200_adj1 gold
ag_gemma_s_n200_v1 · adjudicated100.0200gold/ag_news_200_adj1 gold
ag_qwen_m_n200_v1 · adjudicated103.0200gold/ag_news_200_adj1 gold
ag_llama_m_n200_v1 · adjudicated104.0200gold/ag_news_200_adj1 gold
ag_gemma_l_n200_v1 · adjudicated104.0200gold/ag_news_200_adj1 gold
ag_gemma_m_n200_v1 · adjudicated104.0200gold/ag_news_200_adj1 gold
ag_anthropic_s_n200_v1 · adjudicated104.8200gold/ag_news_200_adj1 gold
ag_mistral_m_n200_v1 · adjudicated107.5200gold/ag_news_200_adj1 gold
ag_qwen_s_n200_v1 · adjudicated116.0200gold/ag_news_200_adj1 gold
ag_anthropic_l_n200_v1 · adjudicated149.8200gold/ag_news_200_adj1 gold
ag_anthropic_m_n200_v1 · adjudicated150.0200gold/ag_news_200_adj1 gold

03on cnn_dailymail_20

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
cnn_bart_l_v1665.820gold/cnn_dailymail_20 gold

04on cnn_dailymail_200

gold/cnn_dailymail_200cnn_bart_l_n200_v1: 697.2cnn_bart_l_xsum_n200_v1: 697.2cnn_bart_m_n200_v1: 697.2cnn_bart_s_n200_v1: 697.2cnn_bart_l_n200_v1 · linux: 697.2cnn_bart_l_xsum_n200_v1 · linux: 697.2cnn_bart_m_n200_v1 · linux: 697.2cnn_bart_s_n200_v1 · linux: 697.2cnn_bart_s_n200_v1 · repeats: 697.2cnn_openai_l_n200_v1: 728.7cnn_openai_m_n200_v1: 728.7cnn_openai_s_n200_v1: 728.7cnn_glm_l_n200_v1: 733.0cnn_glm_s_n200_v1: 735.9cnn_deepseek_m_n200_v1: 736.1cnn_deepseek_s_n200_v1: 736.1cnn_deepseek_l_n200_v1: 736.4cnn_glm_m_n200_v1: 737.8cnn_llama_l_n200_v1: 740.9cnn_llama_s_n200_v1: 740.9cnn_llama_m_n200_v1: 745.1cnn_gemma_s_n200_v1: 750.7cnn_qwen_l_n200_v1: 753.7cnn_qwen_m_n200_v1: 754.0cnn_gemma_l_n200_v1: 755.1cnn_gemma_m_n200_v1: 755.1cnn_mistral_l_n200_v1: 755.7cnn_mistral_s_n200_v1: 755.7cnn_mistral_m_n200_v1: 767.7cnn_qwen_s_n200_v1: 769.7cnn_anthropic_s_n200_v1: 795.0cnn_anthropic_l_n200_v1: 1134.7cnn_anthropic_m_n200_v1: 1134.7silver/cnn_dailymail_200cnn_bart_l_n200_v1 · silver: 697.2cnn_bart_l_xsum_n200_v1 · silver: 697.2cnn_bart_m_n200_v1 · silver: 697.2cnn_bart_s_n200_v1 · silver: 697.2cnn_openai_l_n200_v1 · silver: 728.7cnn_openai_m_n200_v1 · silver: 728.7cnn_openai_s_n200_v1 · silver: 728.7cnn_glm_l_n200_v1 · silver: 733.0cnn_glm_s_n200_v1 · silver: 735.9cnn_deepseek_m_n200_v1 · silver: 736.1cnn_deepseek_s_n200_v1 · silver: 736.1cnn_deepseek_l_n200_v1 · silver: 736.4cnn_glm_m_n200_v1 · silver: 737.8cnn_llama_l_n200_v1 · silver: 740.9cnn_llama_s_n200_v1 · silver: 740.9cnn_llama_m_n200_v1 · silver: 745.1cnn_gemma_s_n200_v1 · silver: 750.7cnn_qwen_l_n200_v1 · silver: 753.7cnn_qwen_m_n200_v1 · silver: 754.0cnn_gemma_l_n200_v1 · silver: 755.1cnn_gemma_m_n200_v1 · silver: 755.1cnn_mistral_l_n200_v1 · silver: 755.7cnn_mistral_s_n200_v1 · silver: 755.7cnn_mistral_m_n200_v1 · silver: 767.7cnn_qwen_s_n200_v1 · silver: 769.7cnn_anthropic_s_n200_v1 · silver: 795.0cnn_anthropic_m_n200_v1 · silver: 1134.7judge judge-summ-pick-v1cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1: 692.5cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1: 692.5cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1: 692.5cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1: 692.5cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1: 724.0cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1: 724.0cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1: 724.0cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1: 728.2cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1: 731.2cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1: 731.3cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1: 731.3cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1: 731.6cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1: 733.1cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1: 736.0cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1: 736.0cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1: 740.4cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1: 745.9cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1: 748.9cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1: 749.2cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1: 750.3cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1: 750.3cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1: 750.8cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1: 750.8cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1: 762.8cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1: 765.0cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1: 789.9cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1: 1127.6cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1: 1127.6judge judge-summ-v1cnn_bart_l_n200_v1 · judged/judge-summ-v1: 695.1cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1: 695.1cnn_bart_m_n200_v1 · judged/judge-summ-v1: 695.1cnn_bart_s_n200_v1 · judged/judge-summ-v1: 695.1cnn_openai_l_n200_v1 · judged/judge-summ-v1: 726.4cnn_openai_m_n200_v1 · judged/judge-summ-v1: 726.4cnn_openai_s_n200_v1 · judged/judge-summ-v1: 726.4cnn_glm_l_n200_v1 · judged/judge-summ-v1: 730.8cnn_glm_s_n200_v1 · judged/judge-summ-v1: 733.8cnn_deepseek_m_n200_v1 · judged/judge-summ-v1: 733.9cnn_deepseek_s_n200_v1 · judged/judge-summ-v1: 733.9cnn_deepseek_l_n200_v1 · judged/judge-summ-v1: 734.2cnn_glm_m_n200_v1 · judged/judge-summ-v1: 735.8cnn_llama_l_n200_v1 · judged/judge-summ-v1: 738.7cnn_llama_s_n200_v1 · judged/judge-summ-v1: 738.7cnn_llama_m_n200_v1 · judged/judge-summ-v1: 743.0cnn_gemma_s_n200_v1 · judged/judge-summ-v1: 748.7cnn_qwen_l_n200_v1 · judged/judge-summ-v1: 751.7cnn_qwen_m_n200_v1 · judged/judge-summ-v1: 752.0cnn_gemma_l_n200_v1 · judged/judge-summ-v1: 753.1cnn_gemma_m_n200_v1 · judged/judge-summ-v1: 753.1cnn_mistral_l_n200_v1 · judged/judge-summ-v1: 753.6cnn_mistral_s_n200_v1 · judged/judge-summ-v1: 753.6cnn_mistral_m_n200_v1 · judged/judge-summ-v1: 765.6cnn_qwen_s_n200_v1 · judged/judge-summ-v1: 767.8cnn_anthropic_s_n200_v1 · judged/judge-summ-v1: 792.7cnn_anthropic_l_n200_v1 · judged/judge-summ-v1: 1131.7cnn_anthropic_m_n200_v1 · judged/judge-summ-v1: 1131.7judge judge-summ-pointwise-v1cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1: 697.2cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1: 736.1judge judge-summ-v2cnn_bart_l_n200_v1 · judged/judge-summ-v2: 697.2cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2: 697.2cnn_bart_m_n200_v1 · judged/judge-summ-v2: 697.2cnn_bart_s_n200_v1 · judged/judge-summ-v2: 697.2cnn_openai_l_n200_v1 · judged/judge-summ-v2: 728.7cnn_openai_m_n200_v1 · judged/judge-summ-v2: 728.7cnn_openai_s_n200_v1 · judged/judge-summ-v2: 728.7cnn_glm_l_n200_v1 · judged/judge-summ-v2: 733.0cnn_glm_s_n200_v1 · judged/judge-summ-v2: 735.9cnn_deepseek_m_n200_v1 · judged/judge-summ-v2: 736.1cnn_deepseek_s_n200_v1 · judged/judge-summ-v2: 736.1cnn_deepseek_l_n200_v1 · judged/judge-summ-v2: 736.4cnn_glm_m_n200_v1 · judged/judge-summ-v2: 737.8cnn_llama_l_n200_v1 · judged/judge-summ-v2: 740.9cnn_llama_s_n200_v1 · judged/judge-summ-v2: 740.9cnn_llama_m_n200_v1 · judged/judge-summ-v2: 745.1cnn_gemma_s_n200_v1 · judged/judge-summ-v2: 750.7cnn_qwen_l_n200_v1 · judged/judge-summ-v2: 753.7cnn_qwen_m_n200_v1 · judged/judge-summ-v2: 754.0cnn_gemma_l_n200_v1 · judged/judge-summ-v2: 755.1cnn_gemma_m_n200_v1 · judged/judge-summ-v2: 755.1cnn_mistral_l_n200_v1 · judged/judge-summ-v2: 755.7cnn_mistral_s_n200_v1 · judged/judge-summ-v2: 755.7cnn_mistral_m_n200_v1 · judged/judge-summ-v2: 767.7cnn_qwen_s_n200_v1 · judged/judge-summ-v2: 769.7cnn_anthropic_s_n200_v1 · judged/judge-summ-v2: 795.0cnn_anthropic_l_n200_v1 · judged/judge-summ-v2: 1134.7cnn_anthropic_m_n200_v1 · judged/judge-summ-v2: 1134.7692.5 · better1134.7

Not drawn: cnn_anthropic_l_n200_v1 · silver, scored against the reference it wrote (its row is in the table).

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
cnn_bart_l_n200_v1697.2200gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1697.2200gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1697.2200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1697.2200gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1 · linux697.2200gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1 · linux697.2200gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1 · linux697.2200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · linux697.2200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · repeats697.2200gold/cnn_dailymail_200 gold
cnn_openai_l_n200_v1728.7200gold/cnn_dailymail_200 gold
cnn_openai_m_n200_v1728.7200gold/cnn_dailymail_200 gold
cnn_openai_s_n200_v1728.7200gold/cnn_dailymail_200 gold
cnn_glm_l_n200_v1733.0200gold/cnn_dailymail_200 gold
cnn_glm_s_n200_v1735.9200gold/cnn_dailymail_200 gold
cnn_deepseek_m_n200_v1736.1200gold/cnn_dailymail_200 gold
cnn_deepseek_s_n200_v1736.1200gold/cnn_dailymail_200 gold
cnn_deepseek_l_n200_v1736.4200gold/cnn_dailymail_200 gold
cnn_glm_m_n200_v1737.8200gold/cnn_dailymail_200 gold
cnn_llama_l_n200_v1740.9200gold/cnn_dailymail_200 gold
cnn_llama_s_n200_v1740.9200gold/cnn_dailymail_200 gold
cnn_llama_m_n200_v1745.1200gold/cnn_dailymail_200 gold
cnn_gemma_s_n200_v1750.7200gold/cnn_dailymail_200 gold
cnn_qwen_l_n200_v1753.7200gold/cnn_dailymail_200 gold
cnn_qwen_m_n200_v1754.0200gold/cnn_dailymail_200 gold
cnn_gemma_l_n200_v1755.1200gold/cnn_dailymail_200 gold
cnn_gemma_m_n200_v1755.1200gold/cnn_dailymail_200 gold
cnn_mistral_l_n200_v1755.7200gold/cnn_dailymail_200 gold
cnn_mistral_s_n200_v1755.7200gold/cnn_dailymail_200 gold
cnn_mistral_m_n200_v1767.7200gold/cnn_dailymail_200 gold
cnn_qwen_s_n200_v1769.7200gold/cnn_dailymail_200 gold
cnn_anthropic_s_n200_v1795.0200gold/cnn_dailymail_200 gold
cnn_anthropic_l_n200_v11134.7200gold/cnn_dailymail_200 gold
cnn_anthropic_m_n200_v11134.7200gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1 · silver697.2200silver/cnn_dailymail_200 silver
cnn_bart_l_xsum_n200_v1 · silver697.2200silver/cnn_dailymail_200 silver
cnn_bart_m_n200_v1 · silver697.2200silver/cnn_dailymail_200 silver
cnn_bart_s_n200_v1 · silver697.2200silver/cnn_dailymail_200 silver
cnn_openai_l_n200_v1 · silver728.7200silver/cnn_dailymail_200 silver
cnn_openai_m_n200_v1 · silver728.7200silver/cnn_dailymail_200 silver
cnn_openai_s_n200_v1 · silver728.7200silver/cnn_dailymail_200 silver
cnn_glm_l_n200_v1 · silver733.0200silver/cnn_dailymail_200 silver
cnn_glm_s_n200_v1 · silver735.9200silver/cnn_dailymail_200 silver
cnn_deepseek_m_n200_v1 · silver736.1200silver/cnn_dailymail_200 silver
cnn_deepseek_s_n200_v1 · silver736.1200silver/cnn_dailymail_200 silver
cnn_deepseek_l_n200_v1 · silver736.4200silver/cnn_dailymail_200 silver
cnn_glm_m_n200_v1 · silver737.8200silver/cnn_dailymail_200 silver
cnn_llama_l_n200_v1 · silver740.9200silver/cnn_dailymail_200 silver
cnn_llama_s_n200_v1 · silver740.9200silver/cnn_dailymail_200 silver
cnn_llama_m_n200_v1 · silver745.1200silver/cnn_dailymail_200 silver
cnn_gemma_s_n200_v1 · silver750.7200silver/cnn_dailymail_200 silver
cnn_qwen_l_n200_v1 · silver753.7200silver/cnn_dailymail_200 silver
cnn_qwen_m_n200_v1 · silver754.0200silver/cnn_dailymail_200 silver
cnn_gemma_l_n200_v1 · silver755.1200silver/cnn_dailymail_200 silver
cnn_gemma_m_n200_v1 · silver755.1200silver/cnn_dailymail_200 silver
cnn_mistral_l_n200_v1 · silver755.7200silver/cnn_dailymail_200 silver
cnn_mistral_s_n200_v1 · silver755.7200silver/cnn_dailymail_200 silver
cnn_mistral_m_n200_v1 · silver767.7200silver/cnn_dailymail_200 silver
cnn_qwen_s_n200_v1 · silver769.7200silver/cnn_dailymail_200 silver
cnn_anthropic_s_n200_v1 · silver795.0200silver/cnn_dailymail_200 silver
cnn_anthropic_l_n200_v1 · silver1134.7200silver/cnn_dailymail_200 silver
cnn_anthropic_m_n200_v1 · silver1134.7200silver/cnn_dailymail_200 silver
cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1692.5198judge judge-summ-pick-v1
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1692.5198judge judge-summ-pick-v1
cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1692.5198judge judge-summ-pick-v1
cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1692.5198judge judge-summ-pick-v1
cnn_bart_l_n200_v1 · judged/judge-summ-v1695.1197judge judge-summ-v1
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1695.1197judge judge-summ-v1
cnn_bart_m_n200_v1 · judged/judge-summ-v1695.1197judge judge-summ-v1
cnn_bart_s_n200_v1 · judged/judge-summ-v1695.1197judge judge-summ-v1
cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1697.2200judge judge-summ-pointwise-v1
cnn_bart_l_n200_v1 · judged/judge-summ-v2697.2200judge judge-summ-v2
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2697.2200judge judge-summ-v2
cnn_bart_m_n200_v1 · judged/judge-summ-v2697.2200judge judge-summ-v2
cnn_bart_s_n200_v1 · judged/judge-summ-v2697.2200judge judge-summ-v2
cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1724.0198judge judge-summ-pick-v1
cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1724.0198judge judge-summ-pick-v1
cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1724.0198judge judge-summ-pick-v1
cnn_openai_l_n200_v1 · judged/judge-summ-v1726.4197judge judge-summ-v1
cnn_openai_m_n200_v1 · judged/judge-summ-v1726.4197judge judge-summ-v1
cnn_openai_s_n200_v1 · judged/judge-summ-v1726.4197judge judge-summ-v1
cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1728.2198judge judge-summ-pick-v1
cnn_openai_l_n200_v1 · judged/judge-summ-v2728.7200judge judge-summ-v2
cnn_openai_m_n200_v1 · judged/judge-summ-v2728.7200judge judge-summ-v2
cnn_openai_s_n200_v1 · judged/judge-summ-v2728.7200judge judge-summ-v2
cnn_glm_l_n200_v1 · judged/judge-summ-v1730.8197judge judge-summ-v1
cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1731.2198judge judge-summ-pick-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1731.3198judge judge-summ-pick-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1731.3198judge judge-summ-pick-v1
cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1731.6198judge judge-summ-pick-v1
cnn_glm_l_n200_v1 · judged/judge-summ-v2733.0200judge judge-summ-v2
cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1733.1198judge judge-summ-pick-v1
cnn_glm_s_n200_v1 · judged/judge-summ-v1733.8197judge judge-summ-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-v1733.9197judge judge-summ-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-v1733.9197judge judge-summ-v1
cnn_deepseek_l_n200_v1 · judged/judge-summ-v1734.2197judge judge-summ-v1
cnn_glm_m_n200_v1 · judged/judge-summ-v1735.8197judge judge-summ-v1
cnn_glm_s_n200_v1 · judged/judge-summ-v2735.9200judge judge-summ-v2
cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1736.0198judge judge-summ-pick-v1
cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1736.0198judge judge-summ-pick-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1736.1200judge judge-summ-pointwise-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-v2736.1200judge judge-summ-v2
cnn_deepseek_s_n200_v1 · judged/judge-summ-v2736.1200judge judge-summ-v2
cnn_deepseek_l_n200_v1 · judged/judge-summ-v2736.4200judge judge-summ-v2
cnn_glm_m_n200_v1 · judged/judge-summ-v2737.8200judge judge-summ-v2
cnn_llama_l_n200_v1 · judged/judge-summ-v1738.7197judge judge-summ-v1
cnn_llama_s_n200_v1 · judged/judge-summ-v1738.7197judge judge-summ-v1
cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1740.4198judge judge-summ-pick-v1
cnn_llama_l_n200_v1 · judged/judge-summ-v2740.9200judge judge-summ-v2
cnn_llama_s_n200_v1 · judged/judge-summ-v2740.9200judge judge-summ-v2
cnn_llama_m_n200_v1 · judged/judge-summ-v1743.0197judge judge-summ-v1
cnn_llama_m_n200_v1 · judged/judge-summ-v2745.1200judge judge-summ-v2
cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1745.9198judge judge-summ-pick-v1
cnn_gemma_s_n200_v1 · judged/judge-summ-v1748.7197judge judge-summ-v1
cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1748.9198judge judge-summ-pick-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1749.2198judge judge-summ-pick-v1
cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1750.3198judge judge-summ-pick-v1
cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1750.3198judge judge-summ-pick-v1
cnn_gemma_s_n200_v1 · judged/judge-summ-v2750.7200judge judge-summ-v2
cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1750.8198judge judge-summ-pick-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1750.8198judge judge-summ-pick-v1
cnn_qwen_l_n200_v1 · judged/judge-summ-v1751.7197judge judge-summ-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-v1752.0197judge judge-summ-v1
cnn_gemma_l_n200_v1 · judged/judge-summ-v1753.1197judge judge-summ-v1
cnn_gemma_m_n200_v1 · judged/judge-summ-v1753.1197judge judge-summ-v1
cnn_mistral_l_n200_v1 · judged/judge-summ-v1753.6197judge judge-summ-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-v1753.6197judge judge-summ-v1
cnn_qwen_l_n200_v1 · judged/judge-summ-v2753.7200judge judge-summ-v2
cnn_qwen_m_n200_v1 · judged/judge-summ-v2754.0200judge judge-summ-v2
cnn_gemma_l_n200_v1 · judged/judge-summ-v2755.1200judge judge-summ-v2
cnn_gemma_m_n200_v1 · judged/judge-summ-v2755.1200judge judge-summ-v2
cnn_mistral_l_n200_v1 · judged/judge-summ-v2755.7200judge judge-summ-v2
cnn_mistral_s_n200_v1 · judged/judge-summ-v2755.7200judge judge-summ-v2
cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1762.8198judge judge-summ-pick-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1765.0198judge judge-summ-pick-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-v1765.6197judge judge-summ-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-v2767.7200judge judge-summ-v2
cnn_qwen_s_n200_v1 · judged/judge-summ-v1767.8197judge judge-summ-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-v2769.7200judge judge-summ-v2
cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1789.9198judge judge-summ-pick-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-v1792.7197judge judge-summ-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-v2795.0200judge judge-summ-v2
cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v11127.6198judge judge-summ-pick-v1
cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v11127.6198judge judge-summ-pick-v1
cnn_anthropic_l_n200_v1 · judged/judge-summ-v11131.7197judge judge-summ-v1
cnn_anthropic_m_n200_v1 · judged/judge-summ-v11131.7197judge judge-summ-v1
cnn_anthropic_l_n200_v1 · judged/judge-summ-v21134.7200judge judge-summ-v2
cnn_anthropic_m_n200_v1 · judged/judge-summ-v21134.7200judge judge-summ-v2

05on cnn_dailymail_dm200

gold/cnn_dailymail_dm200cnn_bart_l_dm200_v1: 846.7cnn_bart_l_xsum_dm200_v1: 846.7cnn_bart_m_dm200_v1: 846.7cnn_bart_s_dm200_v1: 846.7846.2 · better847.2

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
cnn_bart_l_dm200_v1846.7200gold/cnn_dailymail_dm200 gold
cnn_bart_l_xsum_dm200_v1846.7200gold/cnn_dailymail_dm200 gold
cnn_bart_m_dm200_v1846.7200gold/cnn_dailymail_dm200 gold
cnn_bart_s_dm200_v1846.7200gold/cnn_dailymail_dm200 gold

06on dbpedia_280

gold/dbpedia_280db_openai_l_n200_v1: 136.9db_openai_m_n200_v1: 136.9db_openai_s_n200_v1: 136.9db_deepseek_l_n200_v1: 138.7db_deepseek_m_n200_v1: 138.7db_deepseek_s_n200_v1: 138.7db_mistral_l_n200_v1: 141.2db_mistral_s_n200_v1: 141.2db_glm_l_n200_v1: 141.8db_llama_l_n200_v1: 142.3db_llama_s_n200_v1: 142.3db_gemma_s_n200_v1: 143.6db_glm_m_n200_v1: 143.8db_glm_s_n200_v1: 144.2db_qwen_l_n200_v1: 147.6db_gemma_l_n200_v1: 147.6db_gemma_m_n200_v1: 147.6db_qwen_m_n200_v1: 151.9db_llama_m_n200_v1: 152.7db_mistral_m_n200_v1: 153.2db_anthropic_s_n200_v1: 163.0db_qwen_s_n200_v1: 164.9db_anthropic_l_n200_v1: 238.8db_anthropic_m_n200_v1: 238.8136.9 · better238.8

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
db_openai_l_n200_v1136.9280gold/dbpedia_280 gold
db_openai_m_n200_v1136.9280gold/dbpedia_280 gold
db_openai_s_n200_v1136.9280gold/dbpedia_280 gold
db_deepseek_l_n200_v1138.7280gold/dbpedia_280 gold
db_deepseek_m_n200_v1138.7280gold/dbpedia_280 gold
db_deepseek_s_n200_v1138.7280gold/dbpedia_280 gold
db_mistral_l_n200_v1141.2280gold/dbpedia_280 gold
db_mistral_s_n200_v1141.2280gold/dbpedia_280 gold
db_glm_l_n200_v1141.8280gold/dbpedia_280 gold
db_llama_l_n200_v1142.3280gold/dbpedia_280 gold
db_llama_s_n200_v1142.3280gold/dbpedia_280 gold
db_gemma_s_n200_v1143.6280gold/dbpedia_280 gold
db_glm_m_n200_v1143.8280gold/dbpedia_280 gold
db_glm_s_n200_v1144.2280gold/dbpedia_280 gold
db_qwen_l_n200_v1147.6280gold/dbpedia_280 gold
db_gemma_l_n200_v1147.6280gold/dbpedia_280 gold
db_gemma_m_n200_v1147.6280gold/dbpedia_280 gold
db_qwen_m_n200_v1151.9280gold/dbpedia_280 gold
db_llama_m_n200_v1152.7280gold/dbpedia_280 gold
db_mistral_m_n200_v1153.2280gold/dbpedia_280 gold
db_anthropic_s_n200_v1163.0280gold/dbpedia_280 gold
db_qwen_s_n200_v1164.9280gold/dbpedia_280 gold
db_anthropic_l_n200_v1238.8280gold/dbpedia_280 gold
db_anthropic_m_n200_v1238.8280gold/dbpedia_280 gold

07on few_nerd_280

gold/few_nerd_280fn_mistral_l_n200_v1: 114.2fn_mistral_s_n200_v1: 114.2fn_deepseek_l_n200_v1: 114.4fn_deepseek_m_n200_v1: 114.4fn_deepseek_s_n200_v1: 114.4fn_openai_l_n200_v1: 115.4fn_openai_m_n200_v1: 115.4fn_openai_s_n200_v1: 115.4fn_glm_l_n200_v1: 116.4fn_glm_m_n200_v1: 117.4fn_glm_s_n200_v1: 118.8fn_qwen_l_n200_v1: 119.2fn_llama_l_n200_v1: 119.5fn_llama_s_n200_v1: 119.5fn_gemma_s_n200_v1: 122.8fn_anthropic_s_n200_v1: 124.0fn_llama_m_n200_v1: 124.9fn_qwen_m_n200_v1: 124.9fn_mistral_m_n200_v1: 126.2fn_gemma_l_n200_v1: 127.1fn_gemma_m_n200_v1: 127.1fn_qwen_s_n200_v1: 137.9fn_anthropic_l_n200_v1: 168.3fn_anthropic_m_n200_v1: 168.3114.2 · better168.3

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
fn_mistral_l_n200_v1114.2280gold/few_nerd_280 gold
fn_mistral_s_n200_v1114.2280gold/few_nerd_280 gold
fn_deepseek_l_n200_v1114.4280gold/few_nerd_280 gold
fn_deepseek_m_n200_v1114.4280gold/few_nerd_280 gold
fn_deepseek_s_n200_v1114.4280gold/few_nerd_280 gold
fn_openai_l_n200_v1115.4280gold/few_nerd_280 gold
fn_openai_m_n200_v1115.4280gold/few_nerd_280 gold
fn_openai_s_n200_v1115.4280gold/few_nerd_280 gold
fn_glm_l_n200_v1116.4280gold/few_nerd_280 gold
fn_glm_m_n200_v1117.4280gold/few_nerd_280 gold
fn_glm_s_n200_v1118.8280gold/few_nerd_280 gold
fn_qwen_l_n200_v1119.2280gold/few_nerd_280 gold
fn_llama_l_n200_v1119.5280gold/few_nerd_280 gold
fn_llama_s_n200_v1119.5280gold/few_nerd_280 gold
fn_gemma_s_n200_v1122.8280gold/few_nerd_280 gold
fn_anthropic_s_n200_v1124.0280gold/few_nerd_280 gold
fn_llama_m_n200_v1124.9280gold/few_nerd_280 gold
fn_qwen_m_n200_v1124.9280gold/few_nerd_280 gold
fn_mistral_m_n200_v1126.2280gold/few_nerd_280 gold
fn_gemma_l_n200_v1127.1280gold/few_nerd_280 gold
fn_gemma_m_n200_v1127.1280gold/few_nerd_280 gold
fn_qwen_s_n200_v1137.9280gold/few_nerd_280 gold
fn_anthropic_l_n200_v1168.3280gold/few_nerd_280 gold
fn_anthropic_m_n200_v1168.3280gold/few_nerd_280 gold

08on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_stub_dismiss_v1: 1000triage_stub_escalate_v1: 1000triage_stub_dismiss_v1 · rescored 3d3a6afd: 1000triage_stub_escalate_v1 · rescored 3d3a6afd: 1000triage_stub_dismiss_v1 · rescored 5f2c057c: 1000triage_stub_escalate_v1 · rescored 5f2c057c: 1000triage_stub_dismiss_v1 · rescored 77adc97b: 1000triage_stub_escalate_v1 · rescored 77adc97b: 1000triage_stub_dismiss_v1 · rescored cd35dc1f: 1000triage_stub_escalate_v1 · rescored cd35dc1f: 1000triage_local_dgx_gpt_oss_120b_v1: 2618.6triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 2618.6triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 2618.6triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 2618.6triage_local_dgx_gpt_oss_20b_v2: 2720.5triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 2720.5triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 2720.5triage_local_mac_gpt_oss_20b_v1: 2799.6triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 2799.6triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 2799.6triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 2799.6triage_local_dgx_gpt_oss_20b_v1: 2849.9triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 2849.9triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 2849.9triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 2849.9triage_local_dgx_gpt_oss_120b_v2: 2854.7triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 2854.7triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 2854.7triage_local_x3_gpt_oss_20b_v1: 2897.6triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 2897.6triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 2897.6triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 2897.6triage_gemini_35_flash_lite_v1: 2929.6triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 2929.6triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 2929.6triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 2929.6triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 2929.6triage_local_x3_gpt_oss_20b_v2: 2976.1triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 2976.1triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 2976.1triage_local_mac_gpt_oss_20b_v2: 3018.0triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 3018.0triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 3018.0triage_hosted_gpt_oss_120b_v2: 3047triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 3047triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 3047triage_local_mac_gemma4_12b_noreason_v1: 3081triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 3081triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 3081triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 3081triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 3081triage_local_dgx_gemma4_12b_noreason_v1: 3156.5triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 3156.5triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 3156.5triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 3156.5triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 3156.5triage_hosted_gpt_oss_120b_v1: 3253.2triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 3253.2triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 3253.2triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 3253.2triage_local_dgx_nemotron_35_super_120b_v1: 3334.4triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 3334.4triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 3334.4triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 3334.4triage_local_x3_qwen38_27b_v1: 3395.5triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 3395.5triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 3395.5triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 3395.5triage_local_mac_qwen38_27b_v1: 3460.4triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 3460.4triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 3460.4triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 3460.4triage_baseline_pro_v1: 3463.1triage_baseline_pro_v1 · rescored 3d3a6afd: 3463.1triage_baseline_pro_v1 · rescored 5f2c057c: 3463.1triage_baseline_pro_v1 · rescored 77adc97b: 3463.1triage_baseline_pro_v1 · rescored cd35dc1f: 3463.1triage_local_dgx_qwen38_27b_v1: 3538.7triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 3538.7triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 3538.7triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 3538.7triage_baseline_pro_v1 · repeats ×3: 3577.8triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 3577.8triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 3577.8triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 3577.8triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 3577.8triage_deepseek_41flash_v1: 3798.5triage_deepseek_41flash_v1 · rescored 5f2c057c: 3798.5triage_deepseek_41flash_v1 · rescored 77adc97b: 3798.5triage_deepseek_41flash_v1 · rescored cd35dc1f: 3798.5triage_gpt_61_sol_v1 · repeats ×3: 3835.0triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 3835.0triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 3835.0triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 3835.0triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 3835.0triage_gpt_61_sol_v1: 3838.8triage_gpt_61_sol_v1 · rescored 3d3a6afd: 3838.8triage_gpt_61_sol_v1 · rescored 5f2c057c: 3838.8triage_gpt_61_sol_v1 · rescored 77adc97b: 3838.8triage_gpt_61_sol_v1 · rescored cd35dc1f: 3838.8triage_local_dgx_qwen35_122b_v2: 3880.6triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 3880.6triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 3880.6triage_hosted_qwen35_122b_v2: 3944.9triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 3944.9triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 3944.9triage_local_x3_gemma4_31b_v2: 3980.3triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 3980.3triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 3980.3triage_local_dgx_gemma4_31b_v1: 3984.3triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 3984.3triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 3984.3triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 3984.3triage_hosted_gemma4_31b_v2: 3999triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 3999triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 3999triage_local_mac_gemma4_31b_v1: 4000.6triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 4000.6triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 4000.6triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 4000.6triage_hosted_gemma4_31b_v1: 4034.4triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 4034.4triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 4034.4triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 4034.4triage_local_x3_gemma4_31b_v1: 4045.5triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 4045.5triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 4045.5triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 4045.5triage_gpt_6_luna_v1 · repeats ×3: 4067.2triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 4067.2triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 4067.2triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 4067.2triage_local_dgx_gemma4_31b_v2: 4085.8triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 4085.8triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 4085.8triage_local_mac_gemma4_31b_v2: 4099triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 4099triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 4099triage_local_x3_qwen38_27b_v2: 4140.4triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 4140.4triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 4140.4triage_gpt_6_luna_v1: 4166.6triage_gpt_6_luna_v1 · rescored 5f2c057c: 4166.6triage_gpt_6_luna_v1 · rescored 77adc97b: 4166.6triage_gpt_6_luna_v1 · rescored cd35dc1f: 4166.6triage_local_mac_qwen38_27b_v2: 4171.9triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 4171.9triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 4171.9triage_local_dgx_qwen38_27b_v2: 4192.9triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 4192.9triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 4192.9triage_gemini_4_argon_v1: 4236.1triage_gemini_4_argon_v1 · rescored 3d3a6afd: 4236.1triage_gemini_4_argon_v1 · rescored 5f2c057c: 4236.1triage_gemini_4_argon_v1 · rescored 77adc97b: 4236.1triage_gemini_4_argon_v1 · rescored cd35dc1f: 4236.1triage_kimi_k31_pro_v2: 4250.6triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 4250.6triage_kimi_k31_pro_v2 · rescored 5f2c057c: 4250.6triage_local_dgx_qwen35_122b_v1: 4284.4triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 4284.4triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 4284.4triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 4284.4triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 4307.1triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 4307.1triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 4307.1triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 4307.1triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 4307.1triage_hosted_qwen35_122b_v1: 4416.1triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 4416.1triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 4416.1triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 4416.1triage_deepseek_41flash_v2: 4447.3triage_deepseek_41flash_v2 · rescored 3d3a6afd: 4447.3triage_deepseek_41flash_v2 · rescored 5f2c057c: 4447.3triage_kimi_k31_pro_v1: 4579.3triage_kimi_k31_pro_v1 · rescored 5f2c057c: 4579.3triage_kimi_k31_pro_v1 · rescored 77adc97b: 4579.3triage_kimi_k31_pro_v1 · rescored cd35dc1f: 4579.3triage_haiku_55_v1: 4610.4triage_haiku_55_v1 · rescored 3d3a6afd: 4610.4triage_haiku_55_v1 · rescored 5f2c057c: 4610.4triage_haiku_55_v1 · rescored 77adc97b: 4610.4triage_haiku_55_v1 · rescored cd35dc1f: 4610.4triage_haiku_55_v1 · repeats ×3: 4707.3triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 4707.3triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 4707.3triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 4707.3triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 4707.3triage_local_x3_nemotron_35_bolt_e4b_v1: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 4765.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 4765.3triage_local_x3_nemotron_35_lightning_30b_v1: 4803.6triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 4803.6triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 4803.6triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 4803.6triage_local_dgx_nemotron_35_super_120b_v2: 4842.9triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 4842.9triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 4842.9triage_local_dgx_nemotron_35_bolt_e4b_v1: 4845.2triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 4845.2triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 4845.2triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 4845.2triage_local_mac_nemotron_35_lightning_30b_v1: 4856.7triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 4856.7triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 4856.7triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 4856.7triage_local_mac_nemotron_35_bolt_e4b_v1: 4891.7triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 4891.7triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 4891.7triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 4891.7triage_local_dgx_nemotron_35_lightning_30b_v1: 4966.6triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 4966.6triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 4966.6triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 4966.6triage_gpt_6_luna_v2: 5013.0triage_gpt_6_luna_v2 · rescored 3d3a6afd: 5013.0triage_gpt_6_luna_v2 · rescored 5f2c057c: 5013.0triage_local_x3_nemotron_35_bolt_e4b_v2: 5014.6triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 5014.6triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 5014.6triage_mistral_large_4_v1: 5150.2triage_mistral_large_4_v1 · rescored 5f2c057c: 5150.2triage_mistral_large_4_v1 · rescored 77adc97b: 5150.2triage_mistral_large_4_v1 · rescored cd35dc1f: 5150.2triage_mistral_large_4_v2: 5176.6triage_mistral_large_4_v2 · rescored 3d3a6afd: 5176.6triage_mistral_large_4_v2 · rescored 5f2c057c: 5176.6triage_local_dgx_nemotron_35_bolt_e4b_v2: 5202.8triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 5202.8triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 5202.8triage_baseline_pro_noreason_v1: 5222.2triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 5222.2triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 5222.2triage_local_x3_llama31_8b_v1: 5275.6triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 5275.6triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 5275.6triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 5275.6triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 5275.6triage_local_dgx_llama31_8b_v1: 5321.1triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 5321.1triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 5321.1triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 5321.1triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 5321.1triage_local_mac_llama31_8b_v1: 5333.1triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 5333.1triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 5333.1triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 5333.1triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 5333.1triage_local_mac_nemotron_35_bolt_e4b_v2: 5386.8triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 5386.8triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 5386.8triage_local_dgx_llama4_scout_v1: 5431.3triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 5431.3triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 5431.3triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 5431.3triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 5431.3triage_local_mac_phi4_14b_v1: 5531.2triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 5531.2triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 5531.2triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 5531.2triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 5531.2triage_local_x3_phi4_14b_v1: 5594.1triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 5594.1triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 5594.1triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 5594.1triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 5594.1triage_local_dgx_mistral_medium_35_128b_v1: 5611.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 5611.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 5611.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 5611.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 5611.9triage_hosted_qwen3_30b_a3b_v1: 5666.8triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 5666.8triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 5666.8triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 5666.8triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 5666.8triage_local_dgx_phi4_14b_v1: 5700.8triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 5700.8triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 5700.8triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 5700.8triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 5700.8triage_local_x3_qwen3_30b_a3b_v1: 5830.4triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 5830.4triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 5830.4triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 5830.4triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 5830.4triage_local_mac_qwen3_30b_a3b_v1: 5871.6triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 5871.6triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 5871.6triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 5871.6triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 5871.6triage_hosted_mistral_medium_35_v1: 5942.0triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 5942.0triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 5942.0triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 5942.0triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 5942.0triage_local_dgx_qwen3_30b_a3b_v1: 5967.8triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 5967.8triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 5967.8triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 5967.8triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 5967.8triage_local_dgx_ministral_3_14b_v1: 6008.0triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 6008.0triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 6008.0triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 6008.0triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 6008.0triage_local_mac_ministral_3_14b_v1: 6011.1triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 6011.1triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 6011.1triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 6011.1triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 6011.1triage_local_mac_nemotron_35_lightning_30b_v2: 6077.2triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 6077.2triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 6077.2triage_local_dgx_nemotron_35_lightning_30b_v2: 6089.9triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 6089.9triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 6089.9triage_local_x3_nemotron_35_lightning_30b_v2: 6090.2triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 6090.2triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 6090.2triage_sonnet_55_v1: 6103.6triage_sonnet_55_v1 · rescored 3d3a6afd: 6103.6triage_sonnet_55_v1 · rescored 5f2c057c: 6103.6triage_sonnet_55_v1 · rescored 77adc97b: 6103.6triage_sonnet_55_v1 · rescored cd35dc1f: 6103.6triage_local_x3_ministral_3_14b_v1: 6128.5triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 6128.5triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 6128.5triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 6128.5triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 6128.51000 · better6128.5

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
triage_stub_dismiss_v11000.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v11000.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd1000.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd1000.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c1000.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c1000.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b1000.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b1000.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f1000.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f1000.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v12618.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c2618.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b2618.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f2618.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v22720.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd2720.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c2720.565gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v12799.647gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c2799.647gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b2799.647gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f2799.647gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v12849.950gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c2849.950gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b2849.950gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f2849.950gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v22854.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd2854.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c2854.765gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v12897.649gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c2897.649gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b2897.649gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f2897.649gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v12929.665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd2929.665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c2929.665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b2929.665gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f2929.665gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v22976.165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd2976.165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c2976.165gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v23018.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd3018.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c3018.065gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v23047.065gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd3047.065gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c3047.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v13081.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd3081.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c3081.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b3081.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f3081.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v13156.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd3156.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c3156.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b3156.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f3156.565gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v13253.265gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c3253.265gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b3253.265gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f3253.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v13334.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c3334.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b3334.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f3334.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v13395.565gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c3395.565gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b3395.565gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f3395.565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v13460.465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c3460.465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b3460.465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f3460.465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v13463.165gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd3463.165gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c3463.165gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b3463.165gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f3463.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v13538.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c3538.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b3538.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f3538.765gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×33577.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×33577.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×33577.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×33577.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×33577.865gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v13798.565gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c3798.565gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b3798.565gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f3798.565gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×33835.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×33835.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×33835.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×33835.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×33835.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v13838.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd3838.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c3838.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b3838.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f3838.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v23880.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd3880.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c3880.665gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v23944.965gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd3944.965gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c3944.965gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v23980.365gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd3980.365gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c3980.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v13984.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c3984.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b3984.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f3984.365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v23999.065gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd3999.065gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c3999.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v14000.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c4000.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b4000.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f4000.665gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v14034.465gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c4034.465gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b4034.465gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f4034.465gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v14045.565gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c4045.565gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b4045.565gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f4045.565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×34067.265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×34067.265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×34067.265gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×34067.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v24085.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd4085.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c4085.865gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v24099.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd4099.065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c4099.065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v24140.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd4140.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c4140.465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v14166.665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c4166.665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b4166.665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f4166.665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v24171.965gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd4171.965gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c4171.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v24192.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd4192.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c4192.965gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v14236.165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd4236.165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c4236.165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b4236.165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f4236.165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v24250.665gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd4250.665gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c4250.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v14284.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c4284.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b4284.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f4284.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v14307.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd4307.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c4307.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b4307.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f4307.165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v14416.165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c4416.165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b4416.165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f4416.165gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v24447.365gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd4447.365gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c4447.365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v14579.365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c4579.365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b4579.365gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f4579.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v14610.465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd4610.465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c4610.465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b4610.465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f4610.465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×34707.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×34707.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×34707.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×34707.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×34707.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v14765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats4765.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v14803.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c4803.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b4803.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f4803.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v24842.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd4842.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c4842.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v14845.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c4845.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b4845.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f4845.265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v14856.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c4856.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b4856.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f4856.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v14891.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c4891.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b4891.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f4891.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v14966.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c4966.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b4966.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f4966.665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v25013.065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd5013.065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c5013.065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v25014.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd5014.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c5014.665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v15150.265gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c5150.265gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b5150.265gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f5150.265gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v25176.665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd5176.665gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c5176.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v25202.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd5202.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c5202.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v15222.265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd5222.265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c5222.265gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v15275.665gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd5275.665gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c5275.665gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b5275.665gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f5275.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v15321.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd5321.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c5321.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b5321.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f5321.165gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v15333.165gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd5333.165gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c5333.165gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b5333.165gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f5333.165gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v25386.865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd5386.865gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c5386.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v15431.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd5431.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c5431.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b5431.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f5431.365gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v15531.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd5531.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c5531.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b5531.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f5531.265gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v15594.165gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd5594.165gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c5594.165gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b5594.165gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f5594.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v15611.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd5611.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c5611.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b5611.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f5611.965gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v15666.865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd5666.865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c5666.865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b5666.865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f5666.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v15700.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd5700.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c5700.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b5700.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f5700.865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v15830.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd5830.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c5830.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b5830.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f5830.465gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v15871.665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd5871.665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c5871.665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b5871.665gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f5871.665gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v15942.065gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd5942.065gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c5942.065gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b5942.065gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f5942.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v15967.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd5967.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c5967.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b5967.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f5967.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v16008.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd6008.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c6008.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b6008.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f6008.065gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v16011.164gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd6011.164gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c6011.164gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b6011.164gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f6011.164gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v26077.265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd6077.265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c6077.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v26089.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd6089.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c6089.965gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v26090.265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd6090.265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c6090.265gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v16103.665gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd6103.665gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c6103.665gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b6103.665gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f6103.665gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v16128.565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd6128.565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c6128.565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b6128.565gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f6128.565gold/orrery_triage_v1_labelled gold

09on scifact_200

gold/scifact_200sf_deepseek_s_n200_v1: 3765.9sf_deepseek_m_n200_v1: 3766.1sf_llama_l_n200_v1: 3848.3sf_llama_s_n200_v1: 3848.3sf_llama_l_n200_v1 · repeats: 3848.3sf_llama_s_n200_v1 · repeats: 3848.3sf_gemma_s_n200_v1: 3929.5sf_gemma_l_n200_v1: 3933.8sf_gemma_m_n200_v1: 3933.8sf_mistral_s_n200_v1: 3953.6sf_llama_m_n200_v1: 4009.5sf_glm_s_e5first_n200_v1 · pair: 4030.2sf_glm_m_n200_v1: 4035.0sf_glm_s_n200_v1: 4037.0sf_glm_s_bm25first_n200_v1 · pair: 4037.0sf_qwen_s_e5first_n200_v1 · pair: 4181.6sf_qwen_s_n200_v1: 4193.8sf_qwen_s_bm25first_n200_v1 · pair: 4193.8sf_qwen_s_n200_v1 · repeats: 4193.83765.9 · better4193.8

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
sf_deepseek_s_n200_v13765.9200gold/scifact_200 gold
sf_deepseek_m_n200_v13766.1200gold/scifact_200 gold
sf_llama_l_n200_v13848.3200gold/scifact_200 gold
sf_llama_s_n200_v13848.3200gold/scifact_200 gold
sf_llama_l_n200_v1 · repeats3848.3200gold/scifact_200 gold
sf_llama_s_n200_v1 · repeats3848.3200gold/scifact_200 gold
sf_gemma_s_n200_v13929.5200gold/scifact_200 gold
sf_gemma_l_n200_v13933.8200gold/scifact_200 gold
sf_gemma_m_n200_v13933.8200gold/scifact_200 gold
sf_mistral_s_n200_v13953.6200gold/scifact_200 gold
sf_llama_m_n200_v14009.5200gold/scifact_200 gold
sf_glm_s_e5first_n200_v1 · pair4030.2200gold/scifact_200 gold
sf_glm_m_n200_v14035.0200gold/scifact_200 gold
sf_glm_s_n200_v14037.0200gold/scifact_200 gold
sf_glm_s_bm25first_n200_v1 · pair4037.0200gold/scifact_200 gold
sf_qwen_s_e5first_n200_v1 · pair4181.6200gold/scifact_200 gold
sf_qwen_s_n200_v14193.8200gold/scifact_200 gold
sf_qwen_s_bm25first_n200_v1 · pair4193.8200gold/scifact_200 gold
sf_qwen_s_n200_v1 · repeats4193.8200gold/scifact_200 gold

10on summeval_100

gold/summeval_100se_openai_l_n100_v1: 1960.6se_openai_m_n100_v1: 1960.6se_openai_s_n100_v1: 1960.6se_glm_l_n100_v1: 1985.0se_glm_m_n100_v1: 1988se_glm_s_n100_v1: 1988.1se_llama_m_n100_v1: 1990.2se_llama_s_n100_v1: 1990.5se_deepseek_l_n100_v1: 2003.1se_deepseek_s_n100_v1: 2003.1se_deepseek_m_n100_v1: 2003.3se_llama_l_n100_v1: 2003.5se_gemma_l_pubtext_n100_v1 · pair: 2023.1se_gemma_l_pubtextcw_n100_v1 · pair: 2023.1se_gemma_s_n100_v1: 2029.8se_gemma_l_n100_v1: 2034.0se_gemma_m_n100_v1: 2034.0se_gemma_l_n100_pick_v1: 2052.9se_qwen37max_n100_v1: 2053.0se_qwen_m_n100_v1: 2053.0se_mistral_l_n100_v1: 2060.4se_mistral_s_n100_v1: 2060.4se_qwen_s_n100_v1: 2066.0se_mistral_m_n100_v1: 2072.4se_qwen_s_n100_pick_v1: 2083.0se_anthropic_s_n100_v1: 2106.8se_anthropic_l_n100_v1: 2935.5se_anthropic_m_n100_v1: 2935.5se_gemma_l_n100_pairwise_v1: 7447.0se_qwen_s_n100_pairwise_v1: 7590.91960.6 · better7590.9

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_initemsscored against
se_openai_l_n100_v11960.6100gold/summeval_100 gold
se_openai_m_n100_v11960.6100gold/summeval_100 gold
se_openai_s_n100_v11960.6100gold/summeval_100 gold
se_glm_l_n100_v11985.0100gold/summeval_100 gold
se_glm_m_n100_v11988.0100gold/summeval_100 gold
se_glm_s_n100_v11988.1100gold/summeval_100 gold
se_llama_m_n100_v11990.2100gold/summeval_100 gold
se_llama_s_n100_v11990.5100gold/summeval_100 gold
se_deepseek_l_n100_v12003.1100gold/summeval_100 gold
se_deepseek_s_n100_v12003.1100gold/summeval_100 gold
se_deepseek_m_n100_v12003.3100gold/summeval_100 gold
se_llama_l_n100_v12003.5100gold/summeval_100 gold
se_gemma_l_pubtext_n100_v1 · pair2023.1100gold/summeval_100 gold
se_gemma_l_pubtextcw_n100_v1 · pair2023.1100gold/summeval_100 gold
se_gemma_s_n100_v12029.8100gold/summeval_100 gold
se_gemma_l_n100_v12034.0100gold/summeval_100 gold
se_gemma_m_n100_v12034.0100gold/summeval_100 gold
se_gemma_l_n100_pick_v12052.9100gold/summeval_100 gold
se_qwen37max_n100_v12053.0100gold/summeval_100 gold
se_qwen_m_n100_v12053.0100gold/summeval_100 gold
se_mistral_l_n100_v12060.4100gold/summeval_100 gold
se_mistral_s_n100_v12060.4100gold/summeval_100 gold
se_qwen_s_n100_v12066.0100gold/summeval_100 gold
se_mistral_m_n100_v12072.4100gold/summeval_100 gold
se_qwen_s_n100_pick_v12083.0100gold/summeval_100 gold
se_anthropic_s_n100_v12106.8100gold/summeval_100 gold
se_anthropic_l_n100_v12935.5100gold/summeval_100 gold
se_anthropic_m_n100_v12935.5100gold/summeval_100 gold
se_gemma_l_n100_pairwise_v17447.0100gold/summeval_100 gold
se_qwen_s_n100_pairwise_v17590.9100gold/summeval_100 gold