eval-harness

metric

tokens_out

better
lower
kind
cost: what an item cost, from the bill or the tokens; needs no reference
decides
no goal
weighed by
no goal
recorded on
ag_news_200, cnn_dailymail_20, cnn_dailymail_200, cnn_dailymail_dm200, dbpedia_280, few_nerd_280, orrery_triage_v1_labelled, scifact_200, summeval_100

01what it measures, and how

Completion tokens per item, as the provider counted them.

Counted by the provider, not the harness. Most providers bill a reasoning model's thinking as completion tokens; reasoning_tokens says how many of them it was.

02on ag_news_200

gold/ag_news_200ag_qwen_l_n200_v1: 1.37ag_qwen_s_n200_v1: 1.37ag_qwen_m_n200_v1: 1.4ag_deepseek_m_n200_v1: 1.9ag_glm_s_n200_v1: 1.91ag_deepseek_l_n200_v1: 2.215ag_deepseek_s_n200_v1: 2.215ag_llama_l_n200_v1: 2.25ag_llama_s_n200_v1: 2.26ag_gemma_m_n200_v1: 2.31ag_glm_m_n200_v1: 2.32ag_gemma_l_n200_v1: 2.35ag_llama_m_n200_v1: 2.355ag_mistral_m_n200_v1: 2.495ag_mistral_l_n200_v1: 2.555ag_mistral_s_n200_v1: 2.575ag_gemma_s_n200_v1: 3.04ag_anthropic_s_n200_v1: 4.405ag_anthropic_l_n200_v1: 5.04ag_anthropic_m_n200_v1: 5.145ag_openai_s_n200_v1: 5.31ag_openai_l_n200_v1: 5.4ag_openai_m_n200_v1: 5.41ag_glm_l_n200_v1: 9.03gold/ag_news_200_adj1ag_qwen_l_n200_v1 · adjudicated: 1.37ag_qwen_s_n200_v1 · adjudicated: 1.37ag_qwen_m_n200_v1 · adjudicated: 1.4ag_deepseek_m_n200_v1 · adjudicated: 1.9ag_glm_s_n200_v1 · adjudicated: 1.91ag_deepseek_l_n200_v1 · adjudicated: 2.215ag_deepseek_s_n200_v1 · adjudicated: 2.215ag_llama_l_n200_v1 · adjudicated: 2.25ag_llama_s_n200_v1 · adjudicated: 2.26ag_gemma_m_n200_v1 · adjudicated: 2.31ag_glm_m_n200_v1 · adjudicated: 2.32ag_gemma_l_n200_v1 · adjudicated: 2.35ag_llama_m_n200_v1 · adjudicated: 2.355ag_mistral_m_n200_v1 · adjudicated: 2.495ag_mistral_l_n200_v1 · adjudicated: 2.555ag_mistral_s_n200_v1 · adjudicated: 2.575ag_gemma_s_n200_v1 · adjudicated: 3.04ag_anthropic_s_n200_v1 · adjudicated: 4.405ag_anthropic_l_n200_v1 · adjudicated: 5.04ag_anthropic_m_n200_v1 · adjudicated: 5.145ag_openai_s_n200_v1 · adjudicated: 5.31ag_openai_l_n200_v1 · adjudicated: 5.4ag_openai_m_n200_v1 · adjudicated: 5.41ag_glm_l_n200_v1 · adjudicated: 9.031.37 · better9.03

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
ag_qwen_l_n200_v11.370200gold/ag_news_200 gold
ag_qwen_s_n200_v11.370200gold/ag_news_200 gold
ag_qwen_m_n200_v11.400200gold/ag_news_200 gold
ag_deepseek_m_n200_v11.900200gold/ag_news_200 gold
ag_glm_s_n200_v11.910200gold/ag_news_200 gold
ag_deepseek_l_n200_v12.215200gold/ag_news_200 gold
ag_deepseek_s_n200_v12.215200gold/ag_news_200 gold
ag_llama_l_n200_v12.250200gold/ag_news_200 gold
ag_llama_s_n200_v12.260200gold/ag_news_200 gold
ag_gemma_m_n200_v12.310200gold/ag_news_200 gold
ag_glm_m_n200_v12.320200gold/ag_news_200 gold
ag_gemma_l_n200_v12.350200gold/ag_news_200 gold
ag_llama_m_n200_v12.355200gold/ag_news_200 gold
ag_mistral_m_n200_v12.495200gold/ag_news_200 gold
ag_mistral_l_n200_v12.555200gold/ag_news_200 gold
ag_mistral_s_n200_v12.575200gold/ag_news_200 gold
ag_gemma_s_n200_v13.040200gold/ag_news_200 gold
ag_anthropic_s_n200_v14.405200gold/ag_news_200 gold
ag_anthropic_l_n200_v15.040200gold/ag_news_200 gold
ag_anthropic_m_n200_v15.145200gold/ag_news_200 gold
ag_openai_s_n200_v15.310200gold/ag_news_200 gold
ag_openai_l_n200_v15.400200gold/ag_news_200 gold
ag_openai_m_n200_v15.410200gold/ag_news_200 gold
ag_glm_l_n200_v19.030200gold/ag_news_200 gold
ag_qwen_l_n200_v1 · adjudicated1.370200gold/ag_news_200_adj1 gold
ag_qwen_s_n200_v1 · adjudicated1.370200gold/ag_news_200_adj1 gold
ag_qwen_m_n200_v1 · adjudicated1.400200gold/ag_news_200_adj1 gold
ag_deepseek_m_n200_v1 · adjudicated1.900200gold/ag_news_200_adj1 gold
ag_glm_s_n200_v1 · adjudicated1.910200gold/ag_news_200_adj1 gold
ag_deepseek_l_n200_v1 · adjudicated2.215200gold/ag_news_200_adj1 gold
ag_deepseek_s_n200_v1 · adjudicated2.215200gold/ag_news_200_adj1 gold
ag_llama_l_n200_v1 · adjudicated2.250200gold/ag_news_200_adj1 gold
ag_llama_s_n200_v1 · adjudicated2.260200gold/ag_news_200_adj1 gold
ag_gemma_m_n200_v1 · adjudicated2.310200gold/ag_news_200_adj1 gold
ag_glm_m_n200_v1 · adjudicated2.320200gold/ag_news_200_adj1 gold
ag_gemma_l_n200_v1 · adjudicated2.350200gold/ag_news_200_adj1 gold
ag_llama_m_n200_v1 · adjudicated2.355200gold/ag_news_200_adj1 gold
ag_mistral_m_n200_v1 · adjudicated2.495200gold/ag_news_200_adj1 gold
ag_mistral_l_n200_v1 · adjudicated2.555200gold/ag_news_200_adj1 gold
ag_mistral_s_n200_v1 · adjudicated2.575200gold/ag_news_200_adj1 gold
ag_gemma_s_n200_v1 · adjudicated3.040200gold/ag_news_200_adj1 gold
ag_anthropic_s_n200_v1 · adjudicated4.405200gold/ag_news_200_adj1 gold
ag_anthropic_l_n200_v1 · adjudicated5.040200gold/ag_news_200_adj1 gold
ag_anthropic_m_n200_v1 · adjudicated5.145200gold/ag_news_200_adj1 gold
ag_openai_s_n200_v1 · adjudicated5.310200gold/ag_news_200_adj1 gold
ag_openai_l_n200_v1 · adjudicated5.400200gold/ag_news_200_adj1 gold
ag_openai_m_n200_v1 · adjudicated5.410200gold/ag_news_200_adj1 gold
ag_glm_l_n200_v1 · adjudicated9.030200gold/ag_news_200_adj1 gold

03on cnn_dailymail_20

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
cnn_bart_l_v166.220gold/cnn_dailymail_20 gold

04on cnn_dailymail_200

gold/cnn_dailymail_200cnn_bart_l_xsum_n200_v1: 28.2cnn_bart_l_xsum_n200_v1 · linux: 28.2cnn_glm_s_n200_v1: 49.7cnn_gemma_l_n200_v1: 55.1cnn_qwen_s_n200_v1: 55.4cnn_gemma_m_n200_v1: 56.9cnn_openai_l_n200_v1: 57.0cnn_openai_m_n200_v1: 57.2cnn_deepseek_s_n200_v1: 63.6cnn_llama_m_n200_v1: 63.8cnn_mistral_m_n200_v1: 64.1cnn_bart_s_n200_v1: 64.2cnn_bart_s_n200_v1 · linux: 64.2cnn_bart_s_n200_v1 · repeats: 64.2cnn_llama_l_n200_v1: 64.3cnn_openai_s_n200_v1: 64.4cnn_deepseek_l_n200_v1: 65.2cnn_glm_m_n200_v1: 65.3cnn_bart_l_n200_v1: 66.2cnn_bart_l_n200_v1 · linux: 66.2cnn_bart_m_n200_v1: 68.8cnn_bart_m_n200_v1 · linux: 68.8cnn_gemma_s_n200_v1: 69.6cnn_qwen_m_n200_v1: 72.2cnn_mistral_l_n200_v1: 72.4cnn_llama_s_n200_v1: 73.1cnn_qwen_l_n200_v1: 74.6cnn_deepseek_m_n200_v1: 75.6cnn_mistral_s_n200_v1: 77.7cnn_glm_l_n200_v1: 84.7cnn_anthropic_s_n200_v1: 102.1cnn_anthropic_m_n200_v1: 147.0cnn_anthropic_l_n200_v1: 166.1silver/cnn_dailymail_200cnn_bart_l_xsum_n200_v1 · silver: 28.2cnn_glm_s_n200_v1 · silver: 49.7cnn_gemma_l_n200_v1 · silver: 55.1cnn_qwen_s_n200_v1 · silver: 55.4cnn_gemma_m_n200_v1 · silver: 56.9cnn_openai_l_n200_v1 · silver: 57.0cnn_openai_m_n200_v1 · silver: 57.2cnn_deepseek_s_n200_v1 · silver: 63.6cnn_llama_m_n200_v1 · silver: 63.8cnn_mistral_m_n200_v1 · silver: 64.1cnn_bart_s_n200_v1 · silver: 64.2cnn_llama_l_n200_v1 · silver: 64.3cnn_openai_s_n200_v1 · silver: 64.4cnn_deepseek_l_n200_v1 · silver: 65.2cnn_glm_m_n200_v1 · silver: 65.3cnn_bart_l_n200_v1 · silver: 66.2cnn_bart_m_n200_v1 · silver: 68.8cnn_gemma_s_n200_v1 · silver: 69.6cnn_qwen_m_n200_v1 · silver: 72.2cnn_mistral_l_n200_v1 · silver: 72.4cnn_llama_s_n200_v1 · silver: 73.1cnn_qwen_l_n200_v1 · silver: 74.6cnn_deepseek_m_n200_v1 · silver: 75.6cnn_mistral_s_n200_v1 · silver: 77.7cnn_glm_l_n200_v1 · silver: 84.7cnn_anthropic_s_n200_v1 · silver: 102.1cnn_anthropic_m_n200_v1 · silver: 147.0judge judge-summ-pick-v1cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1: 28.1cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1: 49.7cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1: 55.0cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1: 55.3cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1: 56.9cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1: 57.0cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1: 57.2cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1: 63.4cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1: 63.8cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1: 64.0cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1: 64.1cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1: 64.2cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1: 64.4cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1: 65.0cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1: 65.3cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1: 66.2cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1: 68.7cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1: 69.6cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1: 72.2cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1: 72.3cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1: 72.9cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1: 74.5cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1: 75.6cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1: 77.6cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1: 84.9cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1: 101.8cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1: 146.9cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1: 165.8judge judge-summ-v1cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1: 28.1cnn_glm_s_n200_v1 · judged/judge-summ-v1: 49.7cnn_gemma_l_n200_v1 · judged/judge-summ-v1: 55.0cnn_qwen_s_n200_v1 · judged/judge-summ-v1: 55.4cnn_gemma_m_n200_v1 · judged/judge-summ-v1: 56.9cnn_openai_l_n200_v1 · judged/judge-summ-v1: 56.9cnn_openai_m_n200_v1 · judged/judge-summ-v1: 57.1cnn_deepseek_s_n200_v1 · judged/judge-summ-v1: 63.4cnn_llama_m_n200_v1 · judged/judge-summ-v1: 63.8cnn_mistral_m_n200_v1 · judged/judge-summ-v1: 63.9cnn_bart_s_n200_v1 · judged/judge-summ-v1: 64.2cnn_llama_l_n200_v1 · judged/judge-summ-v1: 64.3cnn_openai_s_n200_v1 · judged/judge-summ-v1: 64.3cnn_deepseek_l_n200_v1 · judged/judge-summ-v1: 65.2cnn_glm_m_n200_v1 · judged/judge-summ-v1: 65.4cnn_bart_l_n200_v1 · judged/judge-summ-v1: 66.1cnn_bart_m_n200_v1 · judged/judge-summ-v1: 68.8cnn_gemma_s_n200_v1 · judged/judge-summ-v1: 69.6cnn_qwen_m_n200_v1 · judged/judge-summ-v1: 72.1cnn_mistral_l_n200_v1 · judged/judge-summ-v1: 72.3cnn_llama_s_n200_v1 · judged/judge-summ-v1: 72.9cnn_qwen_l_n200_v1 · judged/judge-summ-v1: 74.6cnn_deepseek_m_n200_v1 · judged/judge-summ-v1: 75.5cnn_mistral_s_n200_v1 · judged/judge-summ-v1: 77.6cnn_glm_l_n200_v1 · judged/judge-summ-v1: 84.9cnn_anthropic_s_n200_v1 · judged/judge-summ-v1: 101.6cnn_anthropic_m_n200_v1 · judged/judge-summ-v1: 146.6cnn_anthropic_l_n200_v1 · judged/judge-summ-v1: 166.1judge judge-summ-v2cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2: 28.2cnn_glm_s_n200_v1 · judged/judge-summ-v2: 49.7cnn_gemma_l_n200_v1 · judged/judge-summ-v2: 55.1cnn_qwen_s_n200_v1 · judged/judge-summ-v2: 55.4cnn_gemma_m_n200_v1 · judged/judge-summ-v2: 56.9cnn_openai_l_n200_v1 · judged/judge-summ-v2: 57.0cnn_openai_m_n200_v1 · judged/judge-summ-v2: 57.2cnn_deepseek_s_n200_v1 · judged/judge-summ-v2: 63.6cnn_llama_m_n200_v1 · judged/judge-summ-v2: 63.8cnn_mistral_m_n200_v1 · judged/judge-summ-v2: 64.1cnn_bart_s_n200_v1 · judged/judge-summ-v2: 64.2cnn_llama_l_n200_v1 · judged/judge-summ-v2: 64.3cnn_openai_s_n200_v1 · judged/judge-summ-v2: 64.4cnn_deepseek_l_n200_v1 · judged/judge-summ-v2: 65.2cnn_glm_m_n200_v1 · judged/judge-summ-v2: 65.3cnn_bart_l_n200_v1 · judged/judge-summ-v2: 66.2cnn_bart_m_n200_v1 · judged/judge-summ-v2: 68.8cnn_gemma_s_n200_v1 · judged/judge-summ-v2: 69.6cnn_qwen_m_n200_v1 · judged/judge-summ-v2: 72.2cnn_mistral_l_n200_v1 · judged/judge-summ-v2: 72.4cnn_llama_s_n200_v1 · judged/judge-summ-v2: 73.1cnn_qwen_l_n200_v1 · judged/judge-summ-v2: 74.6cnn_deepseek_m_n200_v1 · judged/judge-summ-v2: 75.6cnn_mistral_s_n200_v1 · judged/judge-summ-v2: 77.7cnn_glm_l_n200_v1 · judged/judge-summ-v2: 84.7cnn_anthropic_s_n200_v1 · judged/judge-summ-v2: 102.1cnn_anthropic_m_n200_v1 · judged/judge-summ-v2: 147.0cnn_anthropic_l_n200_v1 · judged/judge-summ-v2: 166.1judge judge-summ-pointwise-v1cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1: 66.2cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1: 75.628.1 · better166.1

Not drawn: cnn_anthropic_l_n200_v1 · silver, scored against the reference it wrote (its row is in the table).

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
cnn_bart_l_xsum_n200_v128.2200gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1 · linux28.2200gold/cnn_dailymail_200 gold
cnn_glm_s_n200_v149.7200gold/cnn_dailymail_200 gold
cnn_gemma_l_n200_v155.1200gold/cnn_dailymail_200 gold
cnn_qwen_s_n200_v155.4200gold/cnn_dailymail_200 gold
cnn_gemma_m_n200_v156.9200gold/cnn_dailymail_200 gold
cnn_openai_l_n200_v157.0200gold/cnn_dailymail_200 gold
cnn_openai_m_n200_v157.2200gold/cnn_dailymail_200 gold
cnn_deepseek_s_n200_v163.6200gold/cnn_dailymail_200 gold
cnn_llama_m_n200_v163.8200gold/cnn_dailymail_200 gold
cnn_mistral_m_n200_v164.1200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v164.2200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · linux64.2200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · repeats64.2200gold/cnn_dailymail_200 gold
cnn_llama_l_n200_v164.3200gold/cnn_dailymail_200 gold
cnn_openai_s_n200_v164.4200gold/cnn_dailymail_200 gold
cnn_deepseek_l_n200_v165.2200gold/cnn_dailymail_200 gold
cnn_glm_m_n200_v165.3200gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v166.2200gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1 · linux66.2200gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v168.8200gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1 · linux68.8200gold/cnn_dailymail_200 gold
cnn_gemma_s_n200_v169.6200gold/cnn_dailymail_200 gold
cnn_qwen_m_n200_v172.2200gold/cnn_dailymail_200 gold
cnn_mistral_l_n200_v172.4200gold/cnn_dailymail_200 gold
cnn_llama_s_n200_v173.1200gold/cnn_dailymail_200 gold
cnn_qwen_l_n200_v174.6200gold/cnn_dailymail_200 gold
cnn_deepseek_m_n200_v175.6200gold/cnn_dailymail_200 gold
cnn_mistral_s_n200_v177.7200gold/cnn_dailymail_200 gold
cnn_glm_l_n200_v184.7200gold/cnn_dailymail_200 gold
cnn_anthropic_s_n200_v1102.1200gold/cnn_dailymail_200 gold
cnn_anthropic_m_n200_v1147.0200gold/cnn_dailymail_200 gold
cnn_anthropic_l_n200_v1166.1200gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1 · silver28.2200silver/cnn_dailymail_200 silver
cnn_glm_s_n200_v1 · silver49.7200silver/cnn_dailymail_200 silver
cnn_gemma_l_n200_v1 · silver55.1200silver/cnn_dailymail_200 silver
cnn_qwen_s_n200_v1 · silver55.4200silver/cnn_dailymail_200 silver
cnn_gemma_m_n200_v1 · silver56.9200silver/cnn_dailymail_200 silver
cnn_openai_l_n200_v1 · silver57.0200silver/cnn_dailymail_200 silver
cnn_openai_m_n200_v1 · silver57.2200silver/cnn_dailymail_200 silver
cnn_deepseek_s_n200_v1 · silver63.6200silver/cnn_dailymail_200 silver
cnn_llama_m_n200_v1 · silver63.8200silver/cnn_dailymail_200 silver
cnn_mistral_m_n200_v1 · silver64.1200silver/cnn_dailymail_200 silver
cnn_bart_s_n200_v1 · silver64.2200silver/cnn_dailymail_200 silver
cnn_llama_l_n200_v1 · silver64.3200silver/cnn_dailymail_200 silver
cnn_openai_s_n200_v1 · silver64.4200silver/cnn_dailymail_200 silver
cnn_deepseek_l_n200_v1 · silver65.2200silver/cnn_dailymail_200 silver
cnn_glm_m_n200_v1 · silver65.3200silver/cnn_dailymail_200 silver
cnn_bart_l_n200_v1 · silver66.2200silver/cnn_dailymail_200 silver
cnn_bart_m_n200_v1 · silver68.8200silver/cnn_dailymail_200 silver
cnn_gemma_s_n200_v1 · silver69.6200silver/cnn_dailymail_200 silver
cnn_qwen_m_n200_v1 · silver72.2200silver/cnn_dailymail_200 silver
cnn_mistral_l_n200_v1 · silver72.4200silver/cnn_dailymail_200 silver
cnn_llama_s_n200_v1 · silver73.1200silver/cnn_dailymail_200 silver
cnn_qwen_l_n200_v1 · silver74.6200silver/cnn_dailymail_200 silver
cnn_deepseek_m_n200_v1 · silver75.6200silver/cnn_dailymail_200 silver
cnn_mistral_s_n200_v1 · silver77.7200silver/cnn_dailymail_200 silver
cnn_glm_l_n200_v1 · silver84.7200silver/cnn_dailymail_200 silver
cnn_anthropic_s_n200_v1 · silver102.1200silver/cnn_dailymail_200 silver
cnn_anthropic_m_n200_v1 · silver147.0200silver/cnn_dailymail_200 silver
cnn_anthropic_l_n200_v1 · silver166.1200silver/cnn_dailymail_200 silver
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v128.1198judge judge-summ-pick-v1
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v128.1197judge judge-summ-v1
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v228.2200judge judge-summ-v2
cnn_glm_s_n200_v1 · judged/judge-summ-v149.7197judge judge-summ-v1
cnn_glm_s_n200_v1 · judged/judge-summ-pick-v149.7198judge judge-summ-pick-v1
cnn_glm_s_n200_v1 · judged/judge-summ-v249.7200judge judge-summ-v2
cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v155.0198judge judge-summ-pick-v1
cnn_gemma_l_n200_v1 · judged/judge-summ-v155.0197judge judge-summ-v1
cnn_gemma_l_n200_v1 · judged/judge-summ-v255.1200judge judge-summ-v2
cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v155.3198judge judge-summ-pick-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-v155.4197judge judge-summ-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-v255.4200judge judge-summ-v2
cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v156.9198judge judge-summ-pick-v1
cnn_gemma_m_n200_v1 · judged/judge-summ-v156.9197judge judge-summ-v1
cnn_gemma_m_n200_v1 · judged/judge-summ-v256.9200judge judge-summ-v2
cnn_openai_l_n200_v1 · judged/judge-summ-v156.9197judge judge-summ-v1
cnn_openai_l_n200_v1 · judged/judge-summ-v257.0200judge judge-summ-v2
cnn_openai_l_n200_v1 · judged/judge-summ-pick-v157.0198judge judge-summ-pick-v1
cnn_openai_m_n200_v1 · judged/judge-summ-v157.1197judge judge-summ-v1
cnn_openai_m_n200_v1 · judged/judge-summ-v257.2200judge judge-summ-v2
cnn_openai_m_n200_v1 · judged/judge-summ-pick-v157.2198judge judge-summ-pick-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v163.4198judge judge-summ-pick-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-v163.4197judge judge-summ-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-v263.6200judge judge-summ-v2
cnn_llama_m_n200_v1 · judged/judge-summ-v263.8200judge judge-summ-v2
cnn_llama_m_n200_v1 · judged/judge-summ-v163.8197judge judge-summ-v1
cnn_llama_m_n200_v1 · judged/judge-summ-pick-v163.8198judge judge-summ-pick-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-v163.9197judge judge-summ-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v164.0198judge judge-summ-pick-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-v264.1200judge judge-summ-v2
cnn_bart_s_n200_v1 · judged/judge-summ-pick-v164.1198judge judge-summ-pick-v1
cnn_bart_s_n200_v1 · judged/judge-summ-v264.2200judge judge-summ-v2
cnn_bart_s_n200_v1 · judged/judge-summ-v164.2197judge judge-summ-v1
cnn_llama_l_n200_v1 · judged/judge-summ-pick-v164.2198judge judge-summ-pick-v1
cnn_llama_l_n200_v1 · judged/judge-summ-v164.3197judge judge-summ-v1
cnn_llama_l_n200_v1 · judged/judge-summ-v264.3200judge judge-summ-v2
cnn_openai_s_n200_v1 · judged/judge-summ-v164.3197judge judge-summ-v1
cnn_openai_s_n200_v1 · judged/judge-summ-v264.4200judge judge-summ-v2
cnn_openai_s_n200_v1 · judged/judge-summ-pick-v164.4198judge judge-summ-pick-v1
cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v165.0198judge judge-summ-pick-v1
cnn_deepseek_l_n200_v1 · judged/judge-summ-v165.2197judge judge-summ-v1
cnn_deepseek_l_n200_v1 · judged/judge-summ-v265.2200judge judge-summ-v2
cnn_glm_m_n200_v1 · judged/judge-summ-pick-v165.3198judge judge-summ-pick-v1
cnn_glm_m_n200_v1 · judged/judge-summ-v265.3200judge judge-summ-v2
cnn_glm_m_n200_v1 · judged/judge-summ-v165.4197judge judge-summ-v1
cnn_bart_l_n200_v1 · judged/judge-summ-v166.1197judge judge-summ-v1
cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v166.2200judge judge-summ-pointwise-v1
cnn_bart_l_n200_v1 · judged/judge-summ-v266.2200judge judge-summ-v2
cnn_bart_l_n200_v1 · judged/judge-summ-pick-v166.2198judge judge-summ-pick-v1
cnn_bart_m_n200_v1 · judged/judge-summ-pick-v168.7198judge judge-summ-pick-v1
cnn_bart_m_n200_v1 · judged/judge-summ-v168.8197judge judge-summ-v1
cnn_bart_m_n200_v1 · judged/judge-summ-v268.8200judge judge-summ-v2
cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v169.6198judge judge-summ-pick-v1
cnn_gemma_s_n200_v1 · judged/judge-summ-v269.6200judge judge-summ-v2
cnn_gemma_s_n200_v1 · judged/judge-summ-v169.6197judge judge-summ-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-v172.1197judge judge-summ-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v172.2198judge judge-summ-pick-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-v272.2200judge judge-summ-v2
cnn_mistral_l_n200_v1 · judged/judge-summ-v172.3197judge judge-summ-v1
cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v172.3198judge judge-summ-pick-v1
cnn_mistral_l_n200_v1 · judged/judge-summ-v272.4200judge judge-summ-v2
cnn_llama_s_n200_v1 · judged/judge-summ-pick-v172.9198judge judge-summ-pick-v1
cnn_llama_s_n200_v1 · judged/judge-summ-v172.9197judge judge-summ-v1
cnn_llama_s_n200_v1 · judged/judge-summ-v273.1200judge judge-summ-v2
cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v174.5198judge judge-summ-pick-v1
cnn_qwen_l_n200_v1 · judged/judge-summ-v274.6200judge judge-summ-v2
cnn_qwen_l_n200_v1 · judged/judge-summ-v174.6197judge judge-summ-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-v175.5197judge judge-summ-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v175.6200judge judge-summ-pointwise-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-v275.6200judge judge-summ-v2
cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v175.6198judge judge-summ-pick-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v177.6198judge judge-summ-pick-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-v177.6197judge judge-summ-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-v277.7200judge judge-summ-v2
cnn_glm_l_n200_v1 · judged/judge-summ-v284.7200judge judge-summ-v2
cnn_glm_l_n200_v1 · judged/judge-summ-v184.9197judge judge-summ-v1
cnn_glm_l_n200_v1 · judged/judge-summ-pick-v184.9198judge judge-summ-pick-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-v1101.6197judge judge-summ-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1101.8198judge judge-summ-pick-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-v2102.1200judge judge-summ-v2
cnn_anthropic_m_n200_v1 · judged/judge-summ-v1146.6197judge judge-summ-v1
cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1146.9198judge judge-summ-pick-v1
cnn_anthropic_m_n200_v1 · judged/judge-summ-v2147.0200judge judge-summ-v2
cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1165.8198judge judge-summ-pick-v1
cnn_anthropic_l_n200_v1 · judged/judge-summ-v1166.1197judge judge-summ-v1
cnn_anthropic_l_n200_v1 · judged/judge-summ-v2166.1200judge judge-summ-v2

05on cnn_dailymail_dm200

gold/cnn_dailymail_dm200cnn_bart_l_xsum_dm200_v1: 26.9cnn_bart_s_dm200_v1: 67.2cnn_bart_l_dm200_v1: 87.0cnn_bart_m_dm200_v1: 93.026.9 · better93.0

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
cnn_bart_l_xsum_dm200_v126.9200gold/cnn_dailymail_dm200 gold
cnn_bart_s_dm200_v167.2200gold/cnn_dailymail_dm200 gold
cnn_bart_l_dm200_v187.0200gold/cnn_dailymail_dm200 gold
cnn_bart_m_dm200_v193.0200gold/cnn_dailymail_dm200 gold

06on dbpedia_280

gold/dbpedia_280db_qwen_m_n200_v1: 1.804db_qwen_s_n200_v1: 1.807db_qwen_l_n200_v1: 1.821db_deepseek_m_n200_v1: 2.261db_gemma_l_n200_v1: 2.432db_glm_s_n200_v1: 2.432db_gemma_m_n200_v1: 2.443db_deepseek_s_n200_v1: 2.564db_deepseek_l_n200_v1: 2.636db_llama_l_n200_v1: 2.664db_llama_m_n200_v1: 2.779db_glm_m_n200_v1: 2.804db_llama_s_n200_v1: 2.889db_mistral_s_n200_v1: 2.911db_mistral_l_n200_v1: 2.921db_mistral_m_n200_v1: 2.954db_gemma_s_n200_v1: 3.089db_openai_l_n200_v1: 5.511db_openai_m_n200_v1: 5.518db_anthropic_s_n200_v1: 5.714db_openai_s_n200_v1: 5.736db_anthropic_l_n200_v1: 6.289db_anthropic_m_n200_v1: 6.325db_glm_l_n200_v1: 9.3431.804 · better9.343

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
db_qwen_m_n200_v11.804280gold/dbpedia_280 gold
db_qwen_s_n200_v11.807280gold/dbpedia_280 gold
db_qwen_l_n200_v11.821280gold/dbpedia_280 gold
db_deepseek_m_n200_v12.261280gold/dbpedia_280 gold
db_gemma_l_n200_v12.432280gold/dbpedia_280 gold
db_glm_s_n200_v12.432280gold/dbpedia_280 gold
db_gemma_m_n200_v12.443280gold/dbpedia_280 gold
db_deepseek_s_n200_v12.564280gold/dbpedia_280 gold
db_deepseek_l_n200_v12.636280gold/dbpedia_280 gold
db_llama_l_n200_v12.664280gold/dbpedia_280 gold
db_llama_m_n200_v12.779280gold/dbpedia_280 gold
db_glm_m_n200_v12.804280gold/dbpedia_280 gold
db_llama_s_n200_v12.889280gold/dbpedia_280 gold
db_mistral_s_n200_v12.911280gold/dbpedia_280 gold
db_mistral_l_n200_v12.921280gold/dbpedia_280 gold
db_mistral_m_n200_v12.954280gold/dbpedia_280 gold
db_gemma_s_n200_v13.089280gold/dbpedia_280 gold
db_openai_l_n200_v15.511280gold/dbpedia_280 gold
db_openai_m_n200_v15.518280gold/dbpedia_280 gold
db_anthropic_s_n200_v15.714280gold/dbpedia_280 gold
db_openai_s_n200_v15.736280gold/dbpedia_280 gold
db_anthropic_l_n200_v16.289280gold/dbpedia_280 gold
db_anthropic_m_n200_v16.325280gold/dbpedia_280 gold
db_glm_l_n200_v19.343280gold/dbpedia_280 gold

07on few_nerd_280

gold/few_nerd_280fn_openai_l_n200_v1: 38.2fn_qwen_l_n200_v1: 42.3fn_openai_s_n200_v1: 43.0fn_llama_l_n200_v1: 56.5fn_mistral_m_n200_v1: 57.3fn_deepseek_m_n200_v1: 57.4fn_openai_m_n200_v1: 58.2fn_glm_m_n200_v1: 59.2fn_deepseek_s_n200_v1: 61.5fn_mistral_l_n200_v1: 61.6fn_anthropic_l_n200_v1: 64.9fn_deepseek_l_n200_v1: 65.1fn_llama_s_n200_v1: 65.5fn_glm_s_n200_v1: 65.6fn_mistral_s_n200_v1: 67.7fn_anthropic_m_n200_v1: 68.0fn_qwen_s_n200_v1: 75.0fn_gemma_m_n200_v1: 76.1fn_gemma_l_n200_v1: 79.1fn_qwen_m_n200_v1: 79.4fn_llama_m_n200_v1: 81.7fn_anthropic_s_n200_v1: 86.0fn_gemma_s_n200_v1: 91.9fn_glm_l_n200_v1: 136.038.2 · better136.0

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
fn_openai_l_n200_v138.2280gold/few_nerd_280 gold
fn_qwen_l_n200_v142.3280gold/few_nerd_280 gold
fn_openai_s_n200_v143.0280gold/few_nerd_280 gold
fn_llama_l_n200_v156.5280gold/few_nerd_280 gold
fn_mistral_m_n200_v157.3280gold/few_nerd_280 gold
fn_deepseek_m_n200_v157.4280gold/few_nerd_280 gold
fn_openai_m_n200_v158.2280gold/few_nerd_280 gold
fn_glm_m_n200_v159.2280gold/few_nerd_280 gold
fn_deepseek_s_n200_v161.5280gold/few_nerd_280 gold
fn_mistral_l_n200_v161.6280gold/few_nerd_280 gold
fn_anthropic_l_n200_v164.9280gold/few_nerd_280 gold
fn_deepseek_l_n200_v165.1280gold/few_nerd_280 gold
fn_llama_s_n200_v165.5280gold/few_nerd_280 gold
fn_glm_s_n200_v165.6280gold/few_nerd_280 gold
fn_mistral_s_n200_v167.7280gold/few_nerd_280 gold
fn_anthropic_m_n200_v168.0280gold/few_nerd_280 gold
fn_qwen_s_n200_v175.0280gold/few_nerd_280 gold
fn_gemma_m_n200_v176.1280gold/few_nerd_280 gold
fn_gemma_l_n200_v179.1280gold/few_nerd_280 gold
fn_qwen_m_n200_v179.4280gold/few_nerd_280 gold
fn_llama_m_n200_v181.7280gold/few_nerd_280 gold
fn_anthropic_s_n200_v186.0280gold/few_nerd_280 gold
fn_gemma_s_n200_v191.9280gold/few_nerd_280 gold
fn_glm_l_n200_v1136.0280gold/few_nerd_280 gold

08on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_stub_dismiss_v1: 100triage_stub_escalate_v1: 100triage_stub_dismiss_v1 · rescored 3d3a6afd: 100triage_stub_escalate_v1 · rescored 3d3a6afd: 100triage_stub_dismiss_v1 · rescored 5f2c057c: 100triage_stub_escalate_v1 · rescored 5f2c057c: 100triage_stub_dismiss_v1 · rescored 77adc97b: 100triage_stub_escalate_v1 · rescored 77adc97b: 100triage_stub_dismiss_v1 · rescored cd35dc1f: 100triage_stub_escalate_v1 · rescored cd35dc1f: 100triage_local_mac_nemotron_35_lightning_30b_v2: 188.3triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 188.3triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 188.3triage_gemini_35_flash_lite_v1: 195.5triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 195.5triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 195.5triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 195.5triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 195.5triage_local_dgx_nemotron_35_lightning_30b_v2: 195.9triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 195.9triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 195.9triage_local_dgx_llama4_scout_v1: 197.6triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 197.6triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 197.6triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 197.6triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 197.6triage_local_x3_nemotron_35_lightning_30b_v2: 197.6triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 197.6triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 197.6triage_local_x3_llama31_8b_v1: 214.3triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 214.3triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 214.3triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 214.3triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 214.3triage_local_dgx_llama31_8b_v1: 223.7triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 223.7triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 223.7triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 223.7triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 223.7triage_local_mac_gemma4_12b_noreason_v1: 230.2triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 230.2triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 230.2triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 230.2triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 230.2triage_local_mac_llama31_8b_v1: 231.5triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 231.5triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 231.5triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 231.5triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 231.5triage_local_dgx_gemma4_12b_noreason_v1: 232.3triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 232.3triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 232.3triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 232.3triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 232.3triage_local_x3_gemma4_31b_v2: 238.2triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 238.2triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 238.2triage_hosted_gemma4_31b_v1: 241.5triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 241.5triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 241.5triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 241.5triage_hosted_gemma4_31b_v2: 242.8triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 242.8triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 242.8triage_local_mac_gemma4_31b_v2: 245.6triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 245.6triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 245.6triage_local_dgx_gemma4_31b_v2: 247.0triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 247.0triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 247.0triage_local_x3_gpt_oss_20b_v2: 249.1triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 249.1triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 249.1triage_local_mac_gpt_oss_20b_v2: 252.1triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 252.1triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 252.1triage_local_dgx_gpt_oss_20b_v2: 256.6triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 256.6triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 256.6triage_local_dgx_gpt_oss_120b_v2: 280.6triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 280.6triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 280.6triage_hosted_gpt_oss_120b_v2: 282.9triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 282.9triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 282.9triage_local_x3_nemotron_35_bolt_e4b_v2: 291.2triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 291.2triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 291.2triage_local_dgx_qwen35_122b_v2: 299.1triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 299.1triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 299.1triage_hosted_qwen35_122b_v2: 302triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 302triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 302triage_local_dgx_nemotron_35_bolt_e4b_v2: 302.9triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 302.9triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 302.9triage_deepseek_41flash_v2: 309.2triage_deepseek_41flash_v2 · rescored 3d3a6afd: 309.2triage_deepseek_41flash_v2 · rescored 5f2c057c: 309.2triage_local_mac_nemotron_35_bolt_e4b_v2: 309.4triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 309.4triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 309.4triage_gpt_61_sol_v1: 317.0triage_gpt_61_sol_v1 · rescored 3d3a6afd: 317.0triage_gpt_61_sol_v1 · rescored 5f2c057c: 317.0triage_gpt_61_sol_v1 · rescored 77adc97b: 317.0triage_gpt_61_sol_v1 · rescored cd35dc1f: 317.0triage_gpt_61_sol_v1 · repeats ×3: 324.8triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 324.8triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 324.8triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 324.8triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 324.8triage_local_dgx_mistral_medium_35_128b_v1: 328.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 328.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 328.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 328.9triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 328.9triage_gpt_6_luna_v2: 332.6triage_gpt_6_luna_v2 · rescored 3d3a6afd: 332.6triage_gpt_6_luna_v2 · rescored 5f2c057c: 332.6triage_hosted_mistral_medium_35_v1: 350.1triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 350.1triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 350.1triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 350.1triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 350.1triage_kimi_k31_pro_v2: 354.0triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 354.0triage_kimi_k31_pro_v2 · rescored 5f2c057c: 354.0triage_local_x3_phi4_14b_v1: 355.0triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 355.0triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 355.0triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 355.0triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 355.0triage_local_mac_phi4_14b_v1: 357.2triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 357.2triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 357.2triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 357.2triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 357.2triage_local_dgx_phi4_14b_v1: 363.3triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 363.3triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 363.3triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 363.3triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 363.3triage_baseline_pro_noreason_v1: 367.8triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 367.8triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 367.8triage_local_x3_qwen38_27b_v2: 383triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 383triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 383triage_local_dgx_qwen38_27b_v2: 392.3triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 392.3triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 392.3triage_local_mac_qwen38_27b_v2: 397.5triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 397.5triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 397.5triage_local_x3_qwen3_30b_a3b_v1: 400.4triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 400.4triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 400.4triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 400.4triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 400.4triage_local_dgx_nemotron_35_super_120b_v2: 403.3triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 403.3triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 403.3triage_local_mac_qwen3_30b_a3b_v1: 407.3triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 407.3triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 407.3triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 407.3triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 407.3triage_local_dgx_qwen3_30b_a3b_v1: 416.3triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 416.3triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 416.3triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 416.3triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 416.3triage_hosted_qwen3_30b_a3b_v1: 444.0triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 444.0triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 444.0triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 444.0triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 444.0triage_local_dgx_ministral_3_14b_v1: 481.5triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 481.5triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 481.5triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 481.5triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 481.5triage_local_mac_ministral_3_14b_v1: 485.8triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 485.8triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 485.8triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 485.8triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 485.8triage_local_x3_ministral_3_14b_v1: 489triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 489triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 489triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 489triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 489triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 506.0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 506.0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 506.0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 506.0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 506.0triage_gemini_4_argon_v1: 514.4triage_gemini_4_argon_v1 · rescored 3d3a6afd: 514.4triage_gemini_4_argon_v1 · rescored 5f2c057c: 514.4triage_gemini_4_argon_v1 · rescored 77adc97b: 514.4triage_gemini_4_argon_v1 · rescored cd35dc1f: 514.4triage_gpt_6_luna_v1 · repeats ×3: 554triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 554triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 554triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 554triage_local_dgx_gpt_oss_120b_v1: 575.8triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 575.8triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 575.8triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 575.8triage_gpt_6_luna_v1: 597.3triage_gpt_6_luna_v1 · rescored 5f2c057c: 597.3triage_gpt_6_luna_v1 · rescored 77adc97b: 597.3triage_gpt_6_luna_v1 · rescored cd35dc1f: 597.3triage_hosted_gpt_oss_120b_v1: 601.3triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 601.3triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 601.3triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 601.3triage_local_mac_gpt_oss_20b_v1: 629.6triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 629.6triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 629.6triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 629.6triage_local_dgx_gpt_oss_20b_v1: 688.2triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 688.2triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 688.2triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 688.2triage_local_x3_gpt_oss_20b_v1: 721.7triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 721.7triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 721.7triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 721.7triage_local_x3_qwen38_27b_v1: 992.4triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 992.4triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 992.4triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 992.4triage_local_dgx_qwen38_27b_v1: 996.9triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 996.9triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 996.9triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 996.9triage_kimi_k31_pro_v1: 997.1triage_kimi_k31_pro_v1 · rescored 5f2c057c: 997.1triage_kimi_k31_pro_v1 · rescored 77adc97b: 997.1triage_kimi_k31_pro_v1 · rescored cd35dc1f: 997.1triage_local_mac_qwen38_27b_v1: 1022.8triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 1022.8triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 1022.8triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 1022.8triage_local_dgx_qwen35_122b_v1: 1033.8triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 1033.8triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 1033.8triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 1033.8triage_sonnet_55_v1: 1045.2triage_sonnet_55_v1 · rescored 3d3a6afd: 1045.2triage_sonnet_55_v1 · rescored 5f2c057c: 1045.2triage_sonnet_55_v1 · rescored 77adc97b: 1045.2triage_sonnet_55_v1 · rescored cd35dc1f: 1045.2triage_hosted_qwen35_122b_v1: 1058.3triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 1058.3triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 1058.3triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 1058.3triage_haiku_55_v1: 1073.1triage_haiku_55_v1 · rescored 3d3a6afd: 1073.1triage_haiku_55_v1 · rescored 5f2c057c: 1073.1triage_haiku_55_v1 · rescored 77adc97b: 1073.1triage_haiku_55_v1 · rescored cd35dc1f: 1073.1triage_haiku_55_v1 · repeats ×3: 1107.3triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 1107.3triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 1107.3triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 1107.3triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 1107.3triage_local_dgx_nemotron_35_super_120b_v1: 1179.3triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 1179.3triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 1179.3triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 1179.3triage_local_mac_gemma4_31b_v1: 1270.6triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 1270.6triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 1270.6triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 1270.6triage_baseline_pro_v1: 1327.0triage_baseline_pro_v1 · rescored 3d3a6afd: 1327.0triage_baseline_pro_v1 · rescored 5f2c057c: 1327.0triage_baseline_pro_v1 · rescored 77adc97b: 1327.0triage_baseline_pro_v1 · rescored cd35dc1f: 1327.0triage_baseline_pro_v1 · repeats ×3: 1338.6triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 1338.6triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 1338.6triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 1338.6triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 1338.6triage_local_x3_gemma4_31b_v1: 1340.6triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 1340.6triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 1340.6triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 1340.6triage_deepseek_41flash_v1: 1381.6triage_deepseek_41flash_v1 · rescored 5f2c057c: 1381.6triage_deepseek_41flash_v1 · rescored 77adc97b: 1381.6triage_deepseek_41flash_v1 · rescored cd35dc1f: 1381.6triage_local_dgx_gemma4_31b_v1: 1400.2triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 1400.2triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 1400.2triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 1400.2triage_local_x3_nemotron_35_bolt_e4b_v1: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 1405.3triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 1405.3triage_local_mac_nemotron_35_bolt_e4b_v1: 1428.9triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 1428.9triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 1428.9triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 1428.9triage_local_dgx_nemotron_35_bolt_e4b_v1: 1449.3triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 1449.3triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 1449.3triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 1449.3triage_mistral_large_4_v1: 1553.5triage_mistral_large_4_v1 · rescored 5f2c057c: 1553.5triage_mistral_large_4_v1 · rescored 77adc97b: 1553.5triage_mistral_large_4_v1 · rescored cd35dc1f: 1553.5triage_mistral_large_4_v2: 1583.2triage_mistral_large_4_v2 · rescored 3d3a6afd: 1583.2triage_mistral_large_4_v2 · rescored 5f2c057c: 1583.2triage_local_x3_nemotron_35_lightning_30b_v1: 2548.3triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 2548.3triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 2548.3triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 2548.3triage_local_mac_nemotron_35_lightning_30b_v1: 2627.7triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 2627.7triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 2627.7triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 2627.7triage_local_dgx_nemotron_35_lightning_30b_v1: 2789.3triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 2789.3triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 2789.3triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 2789.3100 · better2789.3

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
triage_stub_dismiss_v1100.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1100.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 3d3a6afd100.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 3d3a6afd100.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 5f2c057c100.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 5f2c057c100.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored 77adc97b100.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored 77adc97b100.065gold/orrery_triage_v1_labelled gold
triage_stub_dismiss_v1 · rescored cd35dc1f100.065gold/orrery_triage_v1_labelled gold
triage_stub_escalate_v1 · rescored cd35dc1f100.065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2188.365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd188.365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c188.365gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1195.565gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd195.565gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c195.565gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b195.565gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f195.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2195.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd195.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c195.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1197.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd197.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c197.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b197.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f197.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2197.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd197.665gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c197.665gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1214.365gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd214.365gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c214.365gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b214.365gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f214.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1223.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd223.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c223.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b223.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f223.765gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1230.265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd230.265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c230.265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b230.265gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f230.265gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1231.565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd231.565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c231.565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b231.565gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f231.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1232.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd232.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c232.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b232.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f232.365gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2238.265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd238.265gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c238.265gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1241.565gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c241.565gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b241.565gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f241.565gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2242.865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd242.865gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c242.865gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2245.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd245.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c245.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2247.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd247.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c247.065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2249.165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd249.165gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c249.165gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2252.165gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd252.165gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c252.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2256.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd256.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c256.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2280.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd280.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c280.665gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2282.965gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd282.965gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c282.965gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2291.265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd291.265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c291.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2299.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd299.165gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c299.165gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2302.065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd302.065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c302.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2302.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd302.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c302.965gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2309.265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd309.265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c309.265gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2309.465gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd309.465gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c309.465gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1317.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd317.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c317.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b317.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f317.065gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×3324.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3324.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3324.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3324.865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3324.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1328.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd328.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c328.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b328.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f328.965gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2332.665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd332.665gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c332.665gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1350.165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd350.165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c350.165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b350.165gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f350.165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2354.065gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd354.065gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c354.065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1355.065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd355.065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c355.065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b355.065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f355.065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1357.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd357.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c357.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b357.265gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f357.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1363.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd363.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c363.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b363.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f363.365gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1367.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd367.865gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c367.865gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2383.065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd383.065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c383.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2392.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd392.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c392.365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2397.565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd397.565gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c397.565gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1400.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd400.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c400.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b400.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f400.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2403.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd403.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c403.365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1407.365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd407.365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c407.365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b407.365gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f407.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1416.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd416.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c416.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b416.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f416.365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1444.065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd444.065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c444.065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b444.065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f444.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1481.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd481.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c481.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b481.565gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f481.565gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1485.864gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd485.864gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c485.864gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b485.864gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f485.864gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1489.065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd489.065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c489.065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b489.065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f489.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1506.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd506.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c506.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b506.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f506.065gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1514.465gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd514.465gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c514.465gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b514.465gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f514.465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×3554.065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3554.065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3554.065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3554.065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1575.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c575.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b575.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f575.865gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1597.365gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c597.365gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b597.365gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f597.365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1601.365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c601.365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b601.365gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f601.365gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1629.647gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c629.647gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b629.647gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f629.647gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1688.250gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c688.250gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b688.250gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f688.250gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1721.749gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c721.749gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b721.749gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f721.749gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1992.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c992.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b992.465gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f992.465gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1996.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c996.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b996.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f996.965gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1997.165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c997.165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b997.165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f997.165gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v11022.865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c1022.865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b1022.865gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f1022.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v11033.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c1033.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b1033.865gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f1033.865gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v11045.265gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd1045.265gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c1045.265gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b1045.265gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f1045.265gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v11058.365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c1058.365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b1058.365gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f1058.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v11073.165gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd1073.165gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c1073.165gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b1073.165gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f1073.165gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×31107.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×31107.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×31107.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×31107.365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×31107.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v11179.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c1179.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b1179.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f1179.365gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v11270.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c1270.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b1270.665gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f1270.665gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v11327.065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd1327.065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c1327.065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b1327.065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f1327.065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×31338.665gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×31338.665gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×31338.665gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×31338.665gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×31338.665gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v11340.665gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c1340.665gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b1340.665gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f1340.665gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v11381.665gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c1381.665gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b1381.665gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f1381.665gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v11400.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c1400.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b1400.265gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f1400.265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v11405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats1405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c1405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats1405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b1405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats1405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f1405.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats1405.365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v11428.965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c1428.965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b1428.965gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f1428.965gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v11449.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c1449.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b1449.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f1449.365gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v11553.565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c1553.565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b1553.565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f1553.565gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v21583.265gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd1583.265gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c1583.265gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v12548.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c2548.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b2548.365gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f2548.365gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v12627.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c2627.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b2627.765gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f2627.765gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v12789.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c2789.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b2789.365gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f2789.365gold/orrery_triage_v1_labelled gold

09on scifact_200

gold/scifact_200sf_llama_m_n200_v1: 99.8sf_deepseek_m_n200_v1: 103.4sf_deepseek_s_n200_v1: 106.5sf_llama_l_n200_v1: 129.0sf_llama_l_n200_v1 · repeats: 132.4sf_glm_m_n200_v1: 136.4sf_glm_s_n200_v1: 144.6sf_glm_s_bm25first_n200_v1 · pair: 145.3sf_glm_s_e5first_n200_v1 · pair: 147.9sf_llama_s_n200_v1: 164.8sf_llama_s_n200_v1 · repeats: 165.5sf_gemma_s_n200_v1: 189.1sf_gemma_m_n200_v1: 231.2sf_mistral_s_n200_v1: 237.9sf_gemma_l_n200_v1: 242.0sf_qwen_s_e5first_n200_v1 · pair: 303.2sf_qwen_s_bm25first_n200_v1 · pair: 311.2sf_qwen_s_n200_v1: 319.8sf_qwen_s_n200_v1 · repeats: 319.899.8 · better319.8

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
sf_llama_m_n200_v199.8200gold/scifact_200 gold
sf_deepseek_m_n200_v1103.4200gold/scifact_200 gold
sf_deepseek_s_n200_v1106.5200gold/scifact_200 gold
sf_llama_l_n200_v1129.0200gold/scifact_200 gold
sf_llama_l_n200_v1 · repeats132.4200gold/scifact_200 gold
sf_glm_m_n200_v1136.4200gold/scifact_200 gold
sf_glm_s_n200_v1144.6200gold/scifact_200 gold
sf_glm_s_bm25first_n200_v1 · pair145.3200gold/scifact_200 gold
sf_glm_s_e5first_n200_v1 · pair147.9200gold/scifact_200 gold
sf_llama_s_n200_v1164.8200gold/scifact_200 gold
sf_llama_s_n200_v1 · repeats165.5200gold/scifact_200 gold
sf_gemma_s_n200_v1189.1200gold/scifact_200 gold
sf_gemma_m_n200_v1231.2200gold/scifact_200 gold
sf_mistral_s_n200_v1237.9200gold/scifact_200 gold
sf_gemma_l_n200_v1242.0200gold/scifact_200 gold
sf_qwen_s_e5first_n200_v1 · pair303.2200gold/scifact_200 gold
sf_qwen_s_bm25first_n200_v1 · pair311.2200gold/scifact_200 gold
sf_qwen_s_n200_v1319.8200gold/scifact_200 gold
sf_qwen_s_n200_v1 · repeats319.8200gold/scifact_200 gold

10on summeval_100

gold/summeval_100se_qwen_s_n100_pick_v1: 7.26se_gemma_l_n100_pick_v1: 9.86se_gemma_l_n100_pairwise_v1: 56se_qwen_s_n100_pairwise_v1: 77.7se_openai_l_n100_v1: 145se_openai_m_n100_v1: 145se_openai_s_n100_v1: 145.2se_qwen37max_n100_v1: 208se_deepseek_m_n100_v1: 209.5se_glm_m_n100_v1: 210.2se_gemma_l_pubtext_n100_v1 · pair: 212se_gemma_m_n100_v1: 212.9se_deepseek_s_n100_v1: 213.7se_anthropic_l_n100_v1: 216se_glm_l_n100_v1: 216.3se_deepseek_l_n100_v1: 216.6se_gemma_l_n100_v1: 217.5se_anthropic_m_n100_v1: 217.7se_gemma_l_pubtextcw_n100_v1 · pair: 217.9se_llama_m_n100_v1: 219.1se_mistral_l_n100_v1: 221se_mistral_m_n100_v1: 221se_anthropic_s_n100_v1: 226se_qwen_s_n100_v1: 231.0se_llama_l_n100_v1: 246.8se_glm_s_n100_v1: 293.5se_gemma_s_n100_v1: 298.8se_llama_s_n100_v1: 322.7se_qwen_m_n100_v1: 333.4se_mistral_s_n100_v1: 374.47.26 · better374.4

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tokens_outitemsscored against
se_qwen_s_n100_pick_v17.260100gold/summeval_100 gold
se_gemma_l_n100_pick_v19.860100gold/summeval_100 gold
se_gemma_l_n100_pairwise_v156.0100gold/summeval_100 gold
se_qwen_s_n100_pairwise_v177.7100gold/summeval_100 gold
se_openai_l_n100_v1145.0100gold/summeval_100 gold
se_openai_m_n100_v1145.0100gold/summeval_100 gold
se_openai_s_n100_v1145.2100gold/summeval_100 gold
se_qwen37max_n100_v1208.0100gold/summeval_100 gold
se_deepseek_m_n100_v1209.5100gold/summeval_100 gold
se_glm_m_n100_v1210.2100gold/summeval_100 gold
se_gemma_l_pubtext_n100_v1 · pair212.0100gold/summeval_100 gold
se_gemma_m_n100_v1212.9100gold/summeval_100 gold
se_deepseek_s_n100_v1213.7100gold/summeval_100 gold
se_anthropic_l_n100_v1216.0100gold/summeval_100 gold
se_glm_l_n100_v1216.3100gold/summeval_100 gold
se_deepseek_l_n100_v1216.6100gold/summeval_100 gold
se_gemma_l_n100_v1217.5100gold/summeval_100 gold
se_anthropic_m_n100_v1217.7100gold/summeval_100 gold
se_gemma_l_pubtextcw_n100_v1 · pair217.9100gold/summeval_100 gold
se_llama_m_n100_v1219.1100gold/summeval_100 gold
se_mistral_l_n100_v1221.0100gold/summeval_100 gold
se_mistral_m_n100_v1221.0100gold/summeval_100 gold
se_anthropic_s_n100_v1226.0100gold/summeval_100 gold
se_qwen_s_n100_v1231.0100gold/summeval_100 gold
se_llama_l_n100_v1246.8100gold/summeval_100 gold
se_glm_s_n100_v1293.5100gold/summeval_100 gold
se_gemma_s_n100_v1298.8100gold/summeval_100 gold
se_llama_s_n100_v1322.7100gold/summeval_100 gold
se_qwen_m_n100_v1333.4100gold/summeval_100 gold
se_mistral_s_n100_v1374.4100gold/summeval_100 gold