eval-harness

metric

cost_usd

better
lower
kind
cost: what an item cost, from the bill or the tokens; needs no reference
decides
no goal
weighed by
Named-entity extraction (Few-NERD), Encyclopedia topic classification (DBpedia), Short news summaries (CNN), News topic classification (AG News), Evidence retrieval for scientific claims (SciFact), A scorer that agrees with experts on summaries (SummEval), Triage decisions on Orrery's error signals
recorded on
ag_news_200, cnn_dailymail_20, cnn_dailymail_200, cnn_dailymail_dm200, dbpedia_280, few_nerd_280, orrery_triage_v1_labelled, scifact_200, smoke_v1, summeval_100

01what it measures, and how

What one item cost, in USD: the provider's bill when the response carries one, else the tokens times the config's prices.

Recorded for every item. When the provider's response carries what it billed (OpenRouter always, a LiteLLM proxy that passes usage.cost through, any OpenAI-compatible endpoint that returns it), that bill is the cost. Otherwise it is tokens_in × usd_per_mtok_in plus tokens_out × usd_per_mtok_out from the arm's config, per million tokens. With neither the cost is unknown and left empty, never 0. Lower is better, and it needs no reference, so arms scored against different references can still be compared on it. evh cost-report shows how far the bills and the price tables differ.

02on ag_news_200

gold/ag_news_200ag_bart_mnli_n200_v1: 0ag_bert_base_fy_n200_v1: 0ag_bert_base_ta_n200_v1: 0ag_bert_mini_n200_v1: 0ag_constant_n200_v1: 0ag_keyword_n200_v1: 0ag_bert_base_fy_n200_v1 · linux: 0ag_bert_base_ta_n200_v1 · linux: 0ag_bert_mini_n200_v1 · linux: 0ag_deepseek_s_n200_v1: 7e-06ag_gemma_m_n200_v1: 7.9e-06ag_mistral_s_n200_v1: 9.5e-06ag_gemma_s_n200_v1: 1.1e-05ag_llama_s_n200_v1: 1.3e-05ag_glm_s_n200_v1: 1.4e-05ag_gemma_l_n200_v1: 1.4e-05ag_qwen_s_n200_v1: 1.8e-05ag_llama_m_n200_v1: 2e-05ag_llama_l_n200_v1: 2.1e-05ag_deepseek_m_n200_v1: 2.2e-05ag_qwen_m_n200_v1: 3.5e-05ag_mistral_m_n200_v1: 4.8e-05ag_mistral_l_n200_v1: 5.2e-05ag_glm_m_n200_v1: 5.3e-05ag_deepseek_l_n200_v1: 5.7e-05ag_qwen_l_n200_v1: 8.2e-05ag_glm_l_n200_v1: 9.3e-05ag_openai_s_n200_v1: 9.5e-05ag_anthropic_s_n200_v1: 0.00013ag_openai_l_n200_v1: 0.00024ag_anthropic_m_n200_v1: 0.00035ag_openai_m_n200_v1: 0.00064ag_anthropic_l_n200_v1: 0.00087gold/ag_news_200_adj1ag_bart_mnli_n200_v1 · adjudicated: 0ag_bert_base_fy_n200_v1 · adjudicated: 0ag_bert_base_ta_n200_v1 · adjudicated: 0ag_bert_mini_n200_v1 · adjudicated: 0ag_constant_n200_v1 · adjudicated: 0ag_keyword_n200_v1 · adjudicated: 0ag_deepseek_s_n200_v1 · adjudicated: 7e-06ag_gemma_m_n200_v1 · adjudicated: 7.9e-06ag_mistral_s_n200_v1 · adjudicated: 9.5e-06ag_gemma_s_n200_v1 · adjudicated: 1.1e-05ag_llama_s_n200_v1 · adjudicated: 1.3e-05ag_glm_s_n200_v1 · adjudicated: 1.4e-05ag_gemma_l_n200_v1 · adjudicated: 1.4e-05ag_qwen_s_n200_v1 · adjudicated: 1.8e-05ag_llama_m_n200_v1 · adjudicated: 2e-05ag_llama_l_n200_v1 · adjudicated: 2.1e-05ag_deepseek_m_n200_v1 · adjudicated: 2.2e-05ag_qwen_m_n200_v1 · adjudicated: 3.5e-05ag_mistral_m_n200_v1 · adjudicated: 4.8e-05ag_mistral_l_n200_v1 · adjudicated: 5.2e-05ag_glm_m_n200_v1 · adjudicated: 5.3e-05ag_deepseek_l_n200_v1 · adjudicated: 5.7e-05ag_qwen_l_n200_v1 · adjudicated: 8.2e-05ag_glm_l_n200_v1 · adjudicated: 9.3e-05ag_openai_s_n200_v1 · adjudicated: 9.5e-05ag_anthropic_s_n200_v1 · adjudicated: 0.00013ag_openai_l_n200_v1 · adjudicated: 0.00024ag_anthropic_m_n200_v1 · adjudicated: 0.00035ag_openai_m_n200_v1 · adjudicated: 0.00064ag_anthropic_l_n200_v1 · adjudicated: 0.000870 · better0.00087

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
ag_bart_mnli_n200_v1$0.000000200gold/ag_news_200 gold
ag_bert_base_fy_n200_v1$0.000000200gold/ag_news_200 gold
ag_bert_base_ta_n200_v1$0.000000200gold/ag_news_200 gold
ag_bert_mini_n200_v1$0.000000200gold/ag_news_200 gold
ag_constant_n200_v1$0.000000200gold/ag_news_200 gold
ag_keyword_n200_v1$0.000000200gold/ag_news_200 gold
ag_bert_base_fy_n200_v1 · linux$0.000000200gold/ag_news_200 gold
ag_bert_base_ta_n200_v1 · linux$0.000000200gold/ag_news_200 gold
ag_bert_mini_n200_v1 · linux$0.000000200gold/ag_news_200 gold
ag_deepseek_s_n200_v1$0.000007200gold/ag_news_200 gold
ag_gemma_m_n200_v1$0.000008200gold/ag_news_200 gold
ag_mistral_s_n200_v1$0.000010200gold/ag_news_200 gold
ag_gemma_s_n200_v1$0.000011200gold/ag_news_200 gold
ag_llama_s_n200_v1$0.000013200gold/ag_news_200 gold
ag_glm_s_n200_v1$0.000014200gold/ag_news_200 gold
ag_gemma_l_n200_v1$0.000014200gold/ag_news_200 gold
ag_qwen_s_n200_v1$0.000018200gold/ag_news_200 gold
ag_llama_m_n200_v1$0.000020200gold/ag_news_200 gold
ag_llama_l_n200_v1$0.000021200gold/ag_news_200 gold
ag_deepseek_m_n200_v1$0.000022200gold/ag_news_200 gold
ag_qwen_m_n200_v1$0.000035200gold/ag_news_200 gold
ag_mistral_m_n200_v1$0.000048200gold/ag_news_200 gold
ag_mistral_l_n200_v1$0.000052200gold/ag_news_200 gold
ag_glm_m_n200_v1$0.000053200gold/ag_news_200 gold
ag_deepseek_l_n200_v1$0.000057200gold/ag_news_200 gold
ag_qwen_l_n200_v1$0.000082200gold/ag_news_200 gold
ag_glm_l_n200_v1$0.000093200gold/ag_news_200 gold
ag_openai_s_n200_v1$0.000095200gold/ag_news_200 gold
ag_anthropic_s_n200_v1$0.000127200gold/ag_news_200 gold
ag_openai_l_n200_v1$0.000244200gold/ag_news_200 gold
ag_anthropic_m_n200_v1$0.000351200gold/ag_news_200 gold
ag_openai_m_n200_v1$0.000636200gold/ag_news_200 gold
ag_anthropic_l_n200_v1$0.000875200gold/ag_news_200 gold
ag_bart_mnli_n200_v1 · adjudicated$0.000000200gold/ag_news_200_adj1 gold
ag_bert_base_fy_n200_v1 · adjudicated$0.000000200gold/ag_news_200_adj1 gold
ag_bert_base_ta_n200_v1 · adjudicated$0.000000200gold/ag_news_200_adj1 gold
ag_bert_mini_n200_v1 · adjudicated$0.000000200gold/ag_news_200_adj1 gold
ag_constant_n200_v1 · adjudicated$0.000000200gold/ag_news_200_adj1 gold
ag_keyword_n200_v1 · adjudicated$0.000000200gold/ag_news_200_adj1 gold
ag_deepseek_s_n200_v1 · adjudicated$0.000007200gold/ag_news_200_adj1 gold
ag_gemma_m_n200_v1 · adjudicated$0.000008200gold/ag_news_200_adj1 gold
ag_mistral_s_n200_v1 · adjudicated$0.000010200gold/ag_news_200_adj1 gold
ag_gemma_s_n200_v1 · adjudicated$0.000011200gold/ag_news_200_adj1 gold
ag_llama_s_n200_v1 · adjudicated$0.000013200gold/ag_news_200_adj1 gold
ag_glm_s_n200_v1 · adjudicated$0.000014200gold/ag_news_200_adj1 gold
ag_gemma_l_n200_v1 · adjudicated$0.000014200gold/ag_news_200_adj1 gold
ag_qwen_s_n200_v1 · adjudicated$0.000018200gold/ag_news_200_adj1 gold
ag_llama_m_n200_v1 · adjudicated$0.000020200gold/ag_news_200_adj1 gold
ag_llama_l_n200_v1 · adjudicated$0.000021200gold/ag_news_200_adj1 gold
ag_deepseek_m_n200_v1 · adjudicated$0.000022200gold/ag_news_200_adj1 gold
ag_qwen_m_n200_v1 · adjudicated$0.000035200gold/ag_news_200_adj1 gold
ag_mistral_m_n200_v1 · adjudicated$0.000048200gold/ag_news_200_adj1 gold
ag_mistral_l_n200_v1 · adjudicated$0.000052200gold/ag_news_200_adj1 gold
ag_glm_m_n200_v1 · adjudicated$0.000053200gold/ag_news_200_adj1 gold
ag_deepseek_l_n200_v1 · adjudicated$0.000057200gold/ag_news_200_adj1 gold
ag_qwen_l_n200_v1 · adjudicated$0.000082200gold/ag_news_200_adj1 gold
ag_glm_l_n200_v1 · adjudicated$0.000093200gold/ag_news_200_adj1 gold
ag_openai_s_n200_v1 · adjudicated$0.000095200gold/ag_news_200_adj1 gold
ag_anthropic_s_n200_v1 · adjudicated$0.000127200gold/ag_news_200_adj1 gold
ag_openai_l_n200_v1 · adjudicated$0.000244200gold/ag_news_200_adj1 gold
ag_anthropic_m_n200_v1 · adjudicated$0.000351200gold/ag_news_200_adj1 gold
ag_openai_m_n200_v1 · adjudicated$0.000636200gold/ag_news_200_adj1 gold
ag_anthropic_l_n200_v1 · adjudicated$0.000875200gold/ag_news_200_adj1 gold

03on cnn_dailymail_20

gold/cnn_dailymail_20cnn_bart_l_v1: 0cnn_lead3_v1: 0-0.5 · better0.5

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
cnn_bart_l_v1$0.00000020gold/cnn_dailymail_20 gold
cnn_lead3_v1$0.00000020gold/cnn_dailymail_20 gold

04on cnn_dailymail_200

gold/cnn_dailymail_200cnn_bart_l_n200_v1: 0cnn_bart_l_xsum_n200_v1: 0cnn_bart_m_n200_v1: 0cnn_bart_s_n200_v1: 0cnn_lead3_n200_v1: 0cnn_bart_l_n200_v1 · linux: 0cnn_bart_l_xsum_n200_v1 · linux: 0cnn_bart_m_n200_v1 · linux: 0cnn_bart_s_n200_v1 · linux: 0cnn_lead3_n200_v1 · linux: 0cnn_bart_s_n200_v1 · repeats: 0cnn_deepseek_s_n200_v1: 6.9e-05cnn_gemma_m_n200_v1: 7.7e-05cnn_mistral_s_n200_v1: 8.9e-05cnn_gemma_s_n200_v1: 0.0001cnn_llama_s_n200_v1: 0.00011cnn_gemma_l_n200_v1: 0.00012cnn_qwen_s_n200_v1: 0.00014cnn_glm_s_n200_v1: 0.00015cnn_llama_m_n200_v1: 0.00016cnn_deepseek_m_n200_v1: 0.00019cnn_llama_l_n200_v1: 0.00019cnn_qwen_m_n200_v1: 0.00033cnn_mistral_m_n200_v1: 0.00044cnn_mistral_l_n200_v1: 0.00049cnn_glm_m_n200_v1: 0.0005cnn_deepseek_l_n200_v1: 0.00069cnn_glm_l_n200_v1: 0.00077cnn_openai_s_n200_v1: 0.00084cnn_qwen_l_n200_v1: 0.00084cnn_anthropic_s_n200_v1: 0.0013cnn_openai_l_n200_v1: 0.0022cnn_anthropic_m_n200_v1: 0.0037cnn_openai_m_n200_v1: 0.0054cnn_anthropic_l_n200_v1: 0.0098silver/cnn_dailymail_200cnn_bart_l_n200_v1 · silver: 0cnn_bart_l_xsum_n200_v1 · silver: 0cnn_bart_m_n200_v1 · silver: 0cnn_bart_s_n200_v1 · silver: 0cnn_lead3_n200_v1 · silver: 0cnn_deepseek_s_n200_v1 · silver: 6.9e-05cnn_gemma_m_n200_v1 · silver: 7.7e-05cnn_mistral_s_n200_v1 · silver: 8.9e-05cnn_gemma_s_n200_v1 · silver: 0.0001cnn_llama_s_n200_v1 · silver: 0.00011cnn_gemma_l_n200_v1 · silver: 0.00012cnn_qwen_s_n200_v1 · silver: 0.00014cnn_glm_s_n200_v1 · silver: 0.00015cnn_llama_m_n200_v1 · silver: 0.00016cnn_deepseek_m_n200_v1 · silver: 0.00019cnn_llama_l_n200_v1 · silver: 0.00019cnn_qwen_m_n200_v1 · silver: 0.00033cnn_mistral_m_n200_v1 · silver: 0.00044cnn_mistral_l_n200_v1 · silver: 0.00049cnn_glm_m_n200_v1 · silver: 0.0005cnn_deepseek_l_n200_v1 · silver: 0.00069cnn_glm_l_n200_v1 · silver: 0.00077cnn_openai_s_n200_v1 · silver: 0.00084cnn_qwen_l_n200_v1 · silver: 0.00084cnn_anthropic_s_n200_v1 · silver: 0.0013cnn_openai_l_n200_v1 · silver: 0.0022cnn_anthropic_m_n200_v1 · silver: 0.0037cnn_openai_m_n200_v1 · silver: 0.0054judge judge-summ-pick-v1cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1: 0cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1: 0cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1: 0cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1: 0cnn_lead3_n200_v1 · judged/judge-summ-pick-v1: 0cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1: 6.8e-05cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1: 7.6e-05cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1: 8.9e-05cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1: 9.9e-05cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1: 0.00011cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1: 0.00012cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1: 0.00013cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1: 0.00015cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1: 0.00016cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1: 0.00019cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1: 0.00019cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1: 0.00033cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1: 0.00043cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1: 0.00048cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1: 0.0005cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1: 0.0007cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1: 0.00077cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1: 0.00083cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1: 0.00084cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1: 0.0013cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1: 0.0022cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1: 0.0037cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1: 0.0053cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1: 0.0098judge judge-summ-pointwise-v1cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1: 0cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1: 0.00019judge judge-summ-v1cnn_bart_l_n200_v1 · judged/judge-summ-v1: 0cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1: 0cnn_bart_m_n200_v1 · judged/judge-summ-v1: 0cnn_bart_s_n200_v1 · judged/judge-summ-v1: 0cnn_lead3_n200_v1 · judged/judge-summ-v1: 0cnn_deepseek_s_n200_v1 · judged/judge-summ-v1: 6.9e-05cnn_gemma_m_n200_v1 · judged/judge-summ-v1: 7.7e-05cnn_mistral_s_n200_v1 · judged/judge-summ-v1: 8.9e-05cnn_gemma_s_n200_v1 · judged/judge-summ-v1: 0.0001cnn_llama_s_n200_v1 · judged/judge-summ-v1: 0.00011cnn_gemma_l_n200_v1 · judged/judge-summ-v1: 0.00012cnn_qwen_s_n200_v1 · judged/judge-summ-v1: 0.00013cnn_glm_s_n200_v1 · judged/judge-summ-v1: 0.00015cnn_llama_m_n200_v1 · judged/judge-summ-v1: 0.00017cnn_deepseek_m_n200_v1 · judged/judge-summ-v1: 0.00019cnn_llama_l_n200_v1 · judged/judge-summ-v1: 0.00019cnn_qwen_m_n200_v1 · judged/judge-summ-v1: 0.00033cnn_mistral_m_n200_v1 · judged/judge-summ-v1: 0.00043cnn_mistral_l_n200_v1 · judged/judge-summ-v1: 0.00049cnn_glm_m_n200_v1 · judged/judge-summ-v1: 0.0005cnn_deepseek_l_n200_v1 · judged/judge-summ-v1: 0.00069cnn_glm_l_n200_v1 · judged/judge-summ-v1: 0.00076cnn_openai_s_n200_v1 · judged/judge-summ-v1: 0.00083cnn_qwen_l_n200_v1 · judged/judge-summ-v1: 0.00084cnn_anthropic_s_n200_v1 · judged/judge-summ-v1: 0.0013cnn_openai_l_n200_v1 · judged/judge-summ-v1: 0.0022cnn_anthropic_m_n200_v1 · judged/judge-summ-v1: 0.0037cnn_openai_m_n200_v1 · judged/judge-summ-v1: 0.0053cnn_anthropic_l_n200_v1 · judged/judge-summ-v1: 0.0098judge judge-summ-v2cnn_bart_l_n200_v1 · judged/judge-summ-v2: 0cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2: 0cnn_bart_m_n200_v1 · judged/judge-summ-v2: 0cnn_bart_s_n200_v1 · judged/judge-summ-v2: 0cnn_lead3_n200_v1 · judged/judge-summ-v2: 0cnn_deepseek_s_n200_v1 · judged/judge-summ-v2: 6.9e-05cnn_gemma_m_n200_v1 · judged/judge-summ-v2: 7.7e-05cnn_mistral_s_n200_v1 · judged/judge-summ-v2: 8.9e-05cnn_gemma_s_n200_v1 · judged/judge-summ-v2: 0.0001cnn_llama_s_n200_v1 · judged/judge-summ-v2: 0.00011cnn_gemma_l_n200_v1 · judged/judge-summ-v2: 0.00012cnn_qwen_s_n200_v1 · judged/judge-summ-v2: 0.00014cnn_glm_s_n200_v1 · judged/judge-summ-v2: 0.00015cnn_llama_m_n200_v1 · judged/judge-summ-v2: 0.00016cnn_deepseek_m_n200_v1 · judged/judge-summ-v2: 0.00019cnn_llama_l_n200_v1 · judged/judge-summ-v2: 0.00019cnn_qwen_m_n200_v1 · judged/judge-summ-v2: 0.00033cnn_mistral_m_n200_v1 · judged/judge-summ-v2: 0.00044cnn_mistral_l_n200_v1 · judged/judge-summ-v2: 0.00049cnn_glm_m_n200_v1 · judged/judge-summ-v2: 0.0005cnn_deepseek_l_n200_v1 · judged/judge-summ-v2: 0.00069cnn_glm_l_n200_v1 · judged/judge-summ-v2: 0.00077cnn_openai_s_n200_v1 · judged/judge-summ-v2: 0.00084cnn_qwen_l_n200_v1 · judged/judge-summ-v2: 0.00084cnn_anthropic_s_n200_v1 · judged/judge-summ-v2: 0.0013cnn_openai_l_n200_v1 · judged/judge-summ-v2: 0.0022cnn_anthropic_m_n200_v1 · judged/judge-summ-v2: 0.0037cnn_openai_m_n200_v1 · judged/judge-summ-v2: 0.0054cnn_anthropic_l_n200_v1 · judged/judge-summ-v2: 0.00980 · better0.0098

Not drawn: cnn_anthropic_l_n200_v1 · silver, scored against the reference it wrote (its row is in the table).

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
cnn_bart_l_n200_v1$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1$0.000000200gold/cnn_dailymail_200 gold
cnn_lead3_n200_v1$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1 · linux$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_l_xsum_n200_v1 · linux$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_m_n200_v1 · linux$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · linux$0.000000200gold/cnn_dailymail_200 gold
cnn_lead3_n200_v1 · linux$0.000000200gold/cnn_dailymail_200 gold
cnn_bart_s_n200_v1 · repeats$0.000000200gold/cnn_dailymail_200 gold
cnn_deepseek_s_n200_v1$0.000069200gold/cnn_dailymail_200 gold
cnn_gemma_m_n200_v1$0.000077200gold/cnn_dailymail_200 gold
cnn_mistral_s_n200_v1$0.000089200gold/cnn_dailymail_200 gold
cnn_gemma_s_n200_v1$0.000100200gold/cnn_dailymail_200 gold
cnn_llama_s_n200_v1$0.000111200gold/cnn_dailymail_200 gold
cnn_gemma_l_n200_v1$0.000119200gold/cnn_dailymail_200 gold
cnn_qwen_s_n200_v1$0.000135200gold/cnn_dailymail_200 gold
cnn_glm_s_n200_v1$0.000152200gold/cnn_dailymail_200 gold
cnn_llama_m_n200_v1$0.000165200gold/cnn_dailymail_200 gold
cnn_deepseek_m_n200_v1$0.000191200gold/cnn_dailymail_200 gold
cnn_llama_l_n200_v1$0.000195200gold/cnn_dailymail_200 gold
cnn_qwen_m_n200_v1$0.000331200gold/cnn_dailymail_200 gold
cnn_mistral_m_n200_v1$0.000435200gold/cnn_dailymail_200 gold
cnn_mistral_l_n200_v1$0.000486200gold/cnn_dailymail_200 gold
cnn_glm_m_n200_v1$0.000497200gold/cnn_dailymail_200 gold
cnn_deepseek_l_n200_v1$0.000694200gold/cnn_dailymail_200 gold
cnn_glm_l_n200_v1$0.000766200gold/cnn_dailymail_200 gold
cnn_openai_s_n200_v1$0.000836200gold/cnn_dailymail_200 gold
cnn_qwen_l_n200_v1$0.000844200gold/cnn_dailymail_200 gold
cnn_anthropic_s_n200_v1$0.001305200gold/cnn_dailymail_200 gold
cnn_openai_l_n200_v1$0.002181200gold/cnn_dailymail_200 gold
cnn_anthropic_m_n200_v1$0.003739200gold/cnn_dailymail_200 gold
cnn_openai_m_n200_v1$0.005358200gold/cnn_dailymail_200 gold
cnn_anthropic_l_n200_v1$0.009826200gold/cnn_dailymail_200 gold
cnn_bart_l_n200_v1 · silver$0.000000200silver/cnn_dailymail_200 silver
cnn_bart_l_xsum_n200_v1 · silver$0.000000200silver/cnn_dailymail_200 silver
cnn_bart_m_n200_v1 · silver$0.000000200silver/cnn_dailymail_200 silver
cnn_bart_s_n200_v1 · silver$0.000000200silver/cnn_dailymail_200 silver
cnn_lead3_n200_v1 · silver$0.000000200silver/cnn_dailymail_200 silver
cnn_deepseek_s_n200_v1 · silver$0.000069200silver/cnn_dailymail_200 silver
cnn_gemma_m_n200_v1 · silver$0.000077200silver/cnn_dailymail_200 silver
cnn_mistral_s_n200_v1 · silver$0.000089200silver/cnn_dailymail_200 silver
cnn_gemma_s_n200_v1 · silver$0.000100200silver/cnn_dailymail_200 silver
cnn_llama_s_n200_v1 · silver$0.000111200silver/cnn_dailymail_200 silver
cnn_gemma_l_n200_v1 · silver$0.000119200silver/cnn_dailymail_200 silver
cnn_qwen_s_n200_v1 · silver$0.000135200silver/cnn_dailymail_200 silver
cnn_glm_s_n200_v1 · silver$0.000152200silver/cnn_dailymail_200 silver
cnn_llama_m_n200_v1 · silver$0.000165200silver/cnn_dailymail_200 silver
cnn_deepseek_m_n200_v1 · silver$0.000191200silver/cnn_dailymail_200 silver
cnn_llama_l_n200_v1 · silver$0.000195200silver/cnn_dailymail_200 silver
cnn_qwen_m_n200_v1 · silver$0.000331200silver/cnn_dailymail_200 silver
cnn_mistral_m_n200_v1 · silver$0.000435200silver/cnn_dailymail_200 silver
cnn_mistral_l_n200_v1 · silver$0.000486200silver/cnn_dailymail_200 silver
cnn_glm_m_n200_v1 · silver$0.000497200silver/cnn_dailymail_200 silver
cnn_deepseek_l_n200_v1 · silver$0.000694200silver/cnn_dailymail_200 silver
cnn_glm_l_n200_v1 · silver$0.000766200silver/cnn_dailymail_200 silver
cnn_openai_s_n200_v1 · silver$0.000836200silver/cnn_dailymail_200 silver
cnn_qwen_l_n200_v1 · silver$0.000844200silver/cnn_dailymail_200 silver
cnn_anthropic_s_n200_v1 · silver$0.001305200silver/cnn_dailymail_200 silver
cnn_openai_l_n200_v1 · silver$0.002181200silver/cnn_dailymail_200 silver
cnn_anthropic_m_n200_v1 · silver$0.003739200silver/cnn_dailymail_200 silver
cnn_openai_m_n200_v1 · silver$0.005358200silver/cnn_dailymail_200 silver
cnn_anthropic_l_n200_v1 · silver$0.009826200silver/cnn_dailymail_200 silver
cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1$0.000000198judge judge-summ-pick-v1
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1$0.000000198judge judge-summ-pick-v1
cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1$0.000000198judge judge-summ-pick-v1
cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1$0.000000198judge judge-summ-pick-v1
cnn_lead3_n200_v1 · judged/judge-summ-pick-v1$0.000000198judge judge-summ-pick-v1
cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1$0.000000200judge judge-summ-pointwise-v1
cnn_bart_l_n200_v1 · judged/judge-summ-v1$0.000000197judge judge-summ-v1
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1$0.000000197judge judge-summ-v1
cnn_bart_m_n200_v1 · judged/judge-summ-v1$0.000000197judge judge-summ-v1
cnn_bart_s_n200_v1 · judged/judge-summ-v1$0.000000197judge judge-summ-v1
cnn_lead3_n200_v1 · judged/judge-summ-v1$0.000000197judge judge-summ-v1
cnn_bart_l_n200_v1 · judged/judge-summ-v2$0.000000200judge judge-summ-v2
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2$0.000000200judge judge-summ-v2
cnn_bart_m_n200_v1 · judged/judge-summ-v2$0.000000200judge judge-summ-v2
cnn_bart_s_n200_v1 · judged/judge-summ-v2$0.000000200judge judge-summ-v2
cnn_lead3_n200_v1 · judged/judge-summ-v2$0.000000200judge judge-summ-v2
cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1$0.000068198judge judge-summ-pick-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-v1$0.000069197judge judge-summ-v1
cnn_deepseek_s_n200_v1 · judged/judge-summ-v2$0.000069200judge judge-summ-v2
cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1$0.000076198judge judge-summ-pick-v1
cnn_gemma_m_n200_v1 · judged/judge-summ-v2$0.000077200judge judge-summ-v2
cnn_gemma_m_n200_v1 · judged/judge-summ-v1$0.000077197judge judge-summ-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1$0.000089198judge judge-summ-pick-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-v1$0.000089197judge judge-summ-v1
cnn_mistral_s_n200_v1 · judged/judge-summ-v2$0.000089200judge judge-summ-v2
cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1$0.000099198judge judge-summ-pick-v1
cnn_gemma_s_n200_v1 · judged/judge-summ-v1$0.000100197judge judge-summ-v1
cnn_gemma_s_n200_v1 · judged/judge-summ-v2$0.000100200judge judge-summ-v2
cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1$0.000111198judge judge-summ-pick-v1
cnn_llama_s_n200_v1 · judged/judge-summ-v1$0.000111197judge judge-summ-v1
cnn_llama_s_n200_v1 · judged/judge-summ-v2$0.000111200judge judge-summ-v2
cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1$0.000118198judge judge-summ-pick-v1
cnn_gemma_l_n200_v1 · judged/judge-summ-v2$0.000119200judge judge-summ-v2
cnn_gemma_l_n200_v1 · judged/judge-summ-v1$0.000119197judge judge-summ-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1$0.000134198judge judge-summ-pick-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-v1$0.000135197judge judge-summ-v1
cnn_qwen_s_n200_v1 · judged/judge-summ-v2$0.000135200judge judge-summ-v2
cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1$0.000151198judge judge-summ-pick-v1
cnn_glm_s_n200_v1 · judged/judge-summ-v1$0.000152197judge judge-summ-v1
cnn_glm_s_n200_v1 · judged/judge-summ-v2$0.000152200judge judge-summ-v2
cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1$0.000163198judge judge-summ-pick-v1
cnn_llama_m_n200_v1 · judged/judge-summ-v2$0.000165200judge judge-summ-v2
cnn_llama_m_n200_v1 · judged/judge-summ-v1$0.000165197judge judge-summ-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1$0.000189198judge judge-summ-pick-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1$0.000191200judge judge-summ-pointwise-v1
cnn_deepseek_m_n200_v1 · judged/judge-summ-v2$0.000191200judge judge-summ-v2
cnn_deepseek_m_n200_v1 · judged/judge-summ-v1$0.000191197judge judge-summ-v1
cnn_llama_l_n200_v1 · judged/judge-summ-v1$0.000193197judge judge-summ-v1
cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1$0.000194198judge judge-summ-pick-v1
cnn_llama_l_n200_v1 · judged/judge-summ-v2$0.000195200judge judge-summ-v2
cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1$0.000329198judge judge-summ-pick-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-v1$0.000330197judge judge-summ-v1
cnn_qwen_m_n200_v1 · judged/judge-summ-v2$0.000331200judge judge-summ-v2
cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1$0.000433198judge judge-summ-pick-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-v1$0.000434197judge judge-summ-v1
cnn_mistral_m_n200_v1 · judged/judge-summ-v2$0.000435200judge judge-summ-v2
cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1$0.000484198judge judge-summ-pick-v1
cnn_mistral_l_n200_v1 · judged/judge-summ-v1$0.000485197judge judge-summ-v1
cnn_mistral_l_n200_v1 · judged/judge-summ-v2$0.000486200judge judge-summ-v2
cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1$0.000496198judge judge-summ-pick-v1
cnn_glm_m_n200_v1 · judged/judge-summ-v1$0.000497197judge judge-summ-v1
cnn_glm_m_n200_v1 · judged/judge-summ-v2$0.000497200judge judge-summ-v2
cnn_deepseek_l_n200_v1 · judged/judge-summ-v1$0.000685197judge judge-summ-v1
cnn_deepseek_l_n200_v1 · judged/judge-summ-v2$0.000694200judge judge-summ-v2
cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1$0.000699198judge judge-summ-pick-v1
cnn_glm_l_n200_v1 · judged/judge-summ-v1$0.000763197judge judge-summ-v1
cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1$0.000765198judge judge-summ-pick-v1
cnn_glm_l_n200_v1 · judged/judge-summ-v2$0.000766200judge judge-summ-v2
cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1$0.000833198judge judge-summ-pick-v1
cnn_openai_s_n200_v1 · judged/judge-summ-v1$0.000834197judge judge-summ-v1
cnn_openai_s_n200_v1 · judged/judge-summ-v2$0.000836200judge judge-summ-v2
cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1$0.000840198judge judge-summ-pick-v1
cnn_qwen_l_n200_v1 · judged/judge-summ-v1$0.000842197judge judge-summ-v1
cnn_qwen_l_n200_v1 · judged/judge-summ-v2$0.000844200judge judge-summ-v2
cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1$0.001299198judge judge-summ-pick-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-v1$0.001301197judge judge-summ-v1
cnn_anthropic_s_n200_v1 · judged/judge-summ-v2$0.001305200judge judge-summ-v2
cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1$0.002169198judge judge-summ-pick-v1
cnn_openai_l_n200_v1 · judged/judge-summ-v1$0.002175197judge judge-summ-v1
cnn_openai_l_n200_v1 · judged/judge-summ-v2$0.002181200judge judge-summ-v2
cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1$0.003724198judge judge-summ-pick-v1
cnn_anthropic_m_n200_v1 · judged/judge-summ-v1$0.003730197judge judge-summ-v1
cnn_anthropic_m_n200_v1 · judged/judge-summ-v2$0.003739200judge judge-summ-v2
cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1$0.005335198judge judge-summ-pick-v1
cnn_openai_m_n200_v1 · judged/judge-summ-v1$0.005346197judge judge-summ-v1
cnn_openai_m_n200_v1 · judged/judge-summ-v2$0.005358200judge judge-summ-v2
cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1$0.009783198judge judge-summ-pick-v1
cnn_anthropic_l_n200_v1 · judged/judge-summ-v1$0.009811197judge judge-summ-v1
cnn_anthropic_l_n200_v1 · judged/judge-summ-v2$0.009826200judge judge-summ-v2

05on cnn_dailymail_dm200

gold/cnn_dailymail_dm200cnn_bart_l_dm200_v1: 0cnn_bart_l_xsum_dm200_v1: 0cnn_bart_m_dm200_v1: 0cnn_bart_s_dm200_v1: 0cnn_lead3_dm200_v1: 0-0.5 · better0.5

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
cnn_bart_l_dm200_v1$0.000000200gold/cnn_dailymail_dm200 gold
cnn_bart_l_xsum_dm200_v1$0.000000200gold/cnn_dailymail_dm200 gold
cnn_bart_m_dm200_v1$0.000000200gold/cnn_dailymail_dm200 gold
cnn_bart_s_dm200_v1$0.000000200gold/cnn_dailymail_dm200 gold
cnn_lead3_dm200_v1$0.000000200gold/cnn_dailymail_dm200 gold

06on dbpedia_280

gold/dbpedia_280db_bart_mnli_n200_v1: 0db_bert_base_fy_n200_v1: 0db_constant_n200_v1: 0db_keyword_n200_v1: 0db_bert_base_fy_n200_v1 · linux: 0db_deepseek_s_n200_v1: 1e-05db_gemma_m_n200_v1: 1.2e-05db_gemma_s_n200_v1: 1.2e-05db_mistral_s_n200_v1: 1.4e-05db_gemma_l_n200_v1: 1.6e-05db_glm_s_n200_v1: 1.8e-05db_qwen_s_n200_v1: 2.6e-05db_llama_s_n200_v1: 2.7e-05db_deepseek_m_n200_v1: 2.8e-05db_llama_m_n200_v1: 3e-05db_llama_l_n200_v1: 3.2e-05db_qwen_m_n200_v1: 5.1e-05db_glm_m_n200_v1: 5.7e-05db_mistral_m_n200_v1: 6.7e-05db_deepseek_l_n200_v1: 7.4e-05db_mistral_l_n200_v1: 7.5e-05db_qwen_l_n200_v1: 0.00012db_glm_l_n200_v1: 0.00013db_openai_s_n200_v1: 0.00013db_anthropic_s_n200_v1: 0.00019db_openai_l_n200_v1: 0.00033db_anthropic_m_n200_v1: 0.00054db_openai_m_n200_v1: 0.00085db_anthropic_l_n200_v1: 0.00140 · better0.0014

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
db_bart_mnli_n200_v1$0.000000280gold/dbpedia_280 gold
db_bert_base_fy_n200_v1$0.000000280gold/dbpedia_280 gold
db_constant_n200_v1$0.000000280gold/dbpedia_280 gold
db_keyword_n200_v1$0.000000280gold/dbpedia_280 gold
db_bert_base_fy_n200_v1 · linux$0.000000280gold/dbpedia_280 gold
db_deepseek_s_n200_v1$0.000010280gold/dbpedia_280 gold
db_gemma_m_n200_v1$0.000012280gold/dbpedia_280 gold
db_gemma_s_n200_v1$0.000012280gold/dbpedia_280 gold
db_mistral_s_n200_v1$0.000014280gold/dbpedia_280 gold
db_gemma_l_n200_v1$0.000016280gold/dbpedia_280 gold
db_glm_s_n200_v1$0.000018280gold/dbpedia_280 gold
db_qwen_s_n200_v1$0.000026280gold/dbpedia_280 gold
db_llama_s_n200_v1$0.000027280gold/dbpedia_280 gold
db_deepseek_m_n200_v1$0.000028280gold/dbpedia_280 gold
db_llama_m_n200_v1$0.000030280gold/dbpedia_280 gold
db_llama_l_n200_v1$0.000032280gold/dbpedia_280 gold
db_qwen_m_n200_v1$0.000051280gold/dbpedia_280 gold
db_glm_m_n200_v1$0.000057280gold/dbpedia_280 gold
db_mistral_m_n200_v1$0.000067280gold/dbpedia_280 gold
db_deepseek_l_n200_v1$0.000074280gold/dbpedia_280 gold
db_mistral_l_n200_v1$0.000075280gold/dbpedia_280 gold
db_qwen_l_n200_v1$0.000122280gold/dbpedia_280 gold
db_glm_l_n200_v1$0.000128280gold/dbpedia_280 gold
db_openai_s_n200_v1$0.000128280gold/dbpedia_280 gold
db_anthropic_s_n200_v1$0.000192280gold/dbpedia_280 gold
db_openai_l_n200_v1$0.000329280gold/dbpedia_280 gold
db_anthropic_m_n200_v1$0.000541280gold/dbpedia_280 gold
db_openai_m_n200_v1$0.000850280gold/dbpedia_280 gold
db_anthropic_l_n200_v1$0.001351280gold/dbpedia_280 gold

07on few_nerd_280

gold/few_nerd_280fn_capitalized_n200_v1: 0fn_gliner_n200_v1: 0fn_nothing_n200_v1: 0fn_span_marker_n200_v1: 0fn_mistral_s_n200_v1: 2.7e-05fn_gemma_m_n200_v1: 3.2e-05fn_deepseek_s_n200_v1: 3.2e-05fn_gemma_s_n200_v1: 3.3e-05fn_llama_s_n200_v1: 3.7e-05fn_gemma_l_n200_v1: 4.8e-05fn_qwen_s_n200_v1: 5.6e-05fn_llama_l_n200_v1: 6.3e-05fn_llama_m_n200_v1: 6.9e-05fn_glm_s_n200_v1: 6.9e-05fn_deepseek_m_n200_v1: 7.1e-05fn_qwen_m_n200_v1: 0.00014fn_mistral_l_n200_v1: 0.00015fn_glm_m_n200_v1: 0.00016fn_mistral_m_n200_v1: 0.00017fn_qwen_l_n200_v1: 0.00026fn_openai_s_n200_v1: 0.00028fn_deepseek_l_n200_v1: 0.0003fn_glm_l_n200_v1: 0.00037fn_anthropic_s_n200_v1: 0.00055fn_openai_l_n200_v1: 0.00061fn_anthropic_m_n200_v1: 0.001fn_openai_m_n200_v1: 0.0023fn_anthropic_l_n200_v1: 0.00250 · better0.0025

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
fn_capitalized_n200_v1$0.000000280gold/few_nerd_280 gold
fn_gliner_n200_v1$0.000000280gold/few_nerd_280 gold
fn_nothing_n200_v1$0.000000280gold/few_nerd_280 gold
fn_span_marker_n200_v1$0.000000280gold/few_nerd_280 gold
fn_mistral_s_n200_v1$0.000027280gold/few_nerd_280 gold
fn_gemma_m_n200_v1$0.000032280gold/few_nerd_280 gold
fn_deepseek_s_n200_v1$0.000032280gold/few_nerd_280 gold
fn_gemma_s_n200_v1$0.000033280gold/few_nerd_280 gold
fn_llama_s_n200_v1$0.000037280gold/few_nerd_280 gold
fn_gemma_l_n200_v1$0.000048280gold/few_nerd_280 gold
fn_qwen_s_n200_v1$0.000056280gold/few_nerd_280 gold
fn_llama_l_n200_v1$0.000063280gold/few_nerd_280 gold
fn_llama_m_n200_v1$0.000069280gold/few_nerd_280 gold
fn_glm_s_n200_v1$0.000069280gold/few_nerd_280 gold
fn_deepseek_m_n200_v1$0.000071280gold/few_nerd_280 gold
fn_qwen_m_n200_v1$0.000142280gold/few_nerd_280 gold
fn_mistral_l_n200_v1$0.000150280gold/few_nerd_280 gold
fn_glm_m_n200_v1$0.000159280gold/few_nerd_280 gold
fn_mistral_m_n200_v1$0.000165280gold/few_nerd_280 gold
fn_qwen_l_n200_v1$0.000258280gold/few_nerd_280 gold
fn_openai_s_n200_v1$0.000280280gold/few_nerd_280 gold
fn_deepseek_l_n200_v1$0.000305280gold/few_nerd_280 gold
fn_glm_l_n200_v1$0.000365280gold/few_nerd_280 gold
fn_anthropic_s_n200_v1$0.000554280gold/few_nerd_280 gold
fn_openai_l_n200_v1$0.000612280gold/few_nerd_280 gold
fn_anthropic_m_n200_v1$0.001016280gold/few_nerd_280 gold
fn_openai_m_n200_v1$0.002324280gold/few_nerd_280 gold
fn_anthropic_l_n200_v1$0.002463280gold/few_nerd_280 gold

08on orrery_triage_v1_labelled

gold/orrery_triage_v1_labelledtriage_local_dgx_gemma4_12b_noreason_v1: 0triage_local_dgx_gemma4_31b_v1: 0triage_local_dgx_gemma4_31b_v2: 0triage_local_dgx_gpt_oss_120b_v1: 0triage_local_dgx_gpt_oss_120b_v2: 0triage_local_dgx_gpt_oss_20b_v1: 0triage_local_dgx_gpt_oss_20b_v2: 0triage_local_dgx_llama31_8b_v1: 0triage_local_dgx_llama4_scout_v1: 0triage_local_dgx_ministral_3_14b_v1: 0triage_local_dgx_mistral_medium_35_128b_v1: 0triage_local_dgx_nemotron_35_bolt_e4b_v1: 0triage_local_dgx_nemotron_35_bolt_e4b_v2: 0triage_local_dgx_nemotron_35_lightning_30b_v1: 0triage_local_dgx_nemotron_35_lightning_30b_v2: 0triage_local_dgx_nemotron_35_super_120b_v1: 0triage_local_dgx_nemotron_35_super_120b_v2: 0triage_local_dgx_phi4_14b_v1: 0triage_local_dgx_qwen35_122b_v1: 0triage_local_dgx_qwen35_122b_v2: 0triage_local_dgx_qwen38_27b_v1: 0triage_local_dgx_qwen38_27b_v2: 0triage_local_dgx_qwen3_30b_a3b_v1: 0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1: 0triage_local_mac_gemma4_12b_noreason_v1: 0triage_local_mac_gemma4_31b_v1: 0triage_local_mac_gemma4_31b_v2: 0triage_local_mac_gpt_oss_20b_v1: 0triage_local_mac_gpt_oss_20b_v2: 0triage_local_mac_llama31_8b_v1: 0triage_local_mac_ministral_3_14b_v1: 0triage_local_mac_nemotron_35_bolt_e4b_v1: 0triage_local_mac_nemotron_35_bolt_e4b_v2: 0triage_local_mac_nemotron_35_lightning_30b_v1: 0triage_local_mac_nemotron_35_lightning_30b_v2: 0triage_local_mac_phi4_14b_v1: 0triage_local_mac_qwen38_27b_v1: 0triage_local_mac_qwen38_27b_v2: 0triage_local_mac_qwen3_30b_a3b_v1: 0triage_local_x3_gemma4_31b_v1: 0triage_local_x3_gemma4_31b_v2: 0triage_local_x3_gpt_oss_20b_v1: 0triage_local_x3_gpt_oss_20b_v2: 0triage_local_x3_llama31_8b_v1: 0triage_local_x3_ministral_3_14b_v1: 0triage_local_x3_nemotron_35_bolt_e4b_v1: 0triage_local_x3_nemotron_35_bolt_e4b_v2: 0triage_local_x3_nemotron_35_lightning_30b_v1: 0triage_local_x3_nemotron_35_lightning_30b_v2: 0triage_local_x3_phi4_14b_v1: 0triage_local_x3_qwen38_27b_v1: 0triage_local_x3_qwen38_27b_v2: 0triage_local_x3_qwen3_30b_a3b_v1: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats: 0triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd: 0triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd: 0triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd: 0triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd: 0triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd: 0triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd: 0triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd: 0triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd: 0triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd: 0triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd: 0triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd: 0triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd: 0triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd: 0triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd: 0triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd: 0triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd: 0triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd: 0triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd: 0triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd: 0triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd: 0triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd: 0triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd: 0triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd: 0triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd: 0triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd: 0triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd: 0triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd: 0triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c: 0triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c: 0triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c: 0triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c: 0triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c: 0triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c: 0triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c: 0triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c: 0triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c: 0triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c: 0triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c: 0triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c: 0triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c: 0triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c: 0triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c: 0triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c: 0triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c: 0triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c: 0triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c: 0triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c: 0triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c: 0triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c: 0triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c: 0triage_local_mac_llama31_8b_v1 · rescored 5f2c057c: 0triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c: 0triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0triage_local_mac_phi4_14b_v1 · rescored 5f2c057c: 0triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c: 0triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c: 0triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c: 0triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c: 0triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c: 0triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c: 0triage_local_x3_llama31_8b_v1 · rescored 5f2c057c: 0triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c: 0triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c: 0triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c: 0triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c: 0triage_local_x3_phi4_14b_v1 · rescored 5f2c057c: 0triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c: 0triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c: 0triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats: 0triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b: 0triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b: 0triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b: 0triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b: 0triage_local_dgx_llama31_8b_v1 · rescored 77adc97b: 0triage_local_dgx_llama4_scout_v1 · rescored 77adc97b: 0triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b: 0triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b: 0triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b: 0triage_local_dgx_phi4_14b_v1 · rescored 77adc97b: 0triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b: 0triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b: 0triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b: 0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b: 0triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b: 0triage_local_mac_gemma4_31b_v1 · rescored 77adc97b: 0triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b: 0triage_local_mac_llama31_8b_v1 · rescored 77adc97b: 0triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b: 0triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0triage_local_mac_phi4_14b_v1 · rescored 77adc97b: 0triage_local_mac_qwen38_27b_v1 · rescored 77adc97b: 0triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b: 0triage_local_x3_gemma4_31b_v1 · rescored 77adc97b: 0triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b: 0triage_local_x3_llama31_8b_v1 · rescored 77adc97b: 0triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b: 0triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b: 0triage_local_x3_phi4_14b_v1 · rescored 77adc97b: 0triage_local_x3_qwen38_27b_v1 · rescored 77adc97b: 0triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats: 0triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f: 0triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f: 0triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f: 0triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f: 0triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f: 0triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f: 0triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f: 0triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f: 0triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f: 0triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f: 0triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f: 0triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f: 0triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f: 0triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f: 0triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f: 0triage_local_mac_llama31_8b_v1 · rescored cd35dc1f: 0triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f: 0triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0triage_local_mac_phi4_14b_v1 · rescored cd35dc1f: 0triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f: 0triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f: 0triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f: 0triage_local_x3_llama31_8b_v1 · rescored cd35dc1f: 0triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f: 0triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f: 0triage_local_x3_phi4_14b_v1 · rescored cd35dc1f: 0triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f: 0triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats: 0triage_deepseek_41flash_v2: 0.00035triage_deepseek_41flash_v2 · rescored 3d3a6afd: 0.00035triage_deepseek_41flash_v2 · rescored 5f2c057c: 0.00035triage_hosted_gpt_oss_120b_v2: 0.00063triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd: 0.00063triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c: 0.00063triage_gpt_6_luna_v2: 0.00067triage_gpt_6_luna_v2 · rescored 3d3a6afd: 0.00067triage_gpt_6_luna_v2 · rescored 5f2c057c: 0.00067triage_gpt_6_luna_v1 · repeats ×3: 0.00068triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3: 0.00068triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3: 0.00068triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3: 0.00068triage_hosted_qwen3_30b_a3b_v1: 0.0007triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd: 0.0007triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c: 0.0007triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b: 0.0007triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f: 0.0007triage_gpt_6_luna_v1: 0.00072triage_gpt_6_luna_v1 · rescored 5f2c057c: 0.00072triage_gpt_6_luna_v1 · rescored 77adc97b: 0.00072triage_gpt_6_luna_v1 · rescored cd35dc1f: 0.00072triage_hosted_gpt_oss_120b_v1: 0.00085triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c: 0.00085triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b: 0.00085triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f: 0.00085triage_deepseek_41flash_v1: 0.00086triage_deepseek_41flash_v1 · rescored 5f2c057c: 0.00086triage_deepseek_41flash_v1 · rescored 77adc97b: 0.00086triage_deepseek_41flash_v1 · rescored cd35dc1f: 0.00086triage_hosted_gemma4_31b_v2: 0.0009triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd: 0.0009triage_hosted_gemma4_31b_v2 · rescored 5f2c057c: 0.0009triage_hosted_gemma4_31b_v1: 0.0009triage_hosted_gemma4_31b_v1 · rescored 5f2c057c: 0.0009triage_hosted_gemma4_31b_v1 · rescored 77adc97b: 0.0009triage_hosted_gemma4_31b_v1 · rescored cd35dc1f: 0.0009triage_haiku_55_v1: 0.001triage_haiku_55_v1 · rescored 3d3a6afd: 0.001triage_haiku_55_v1 · rescored 5f2c057c: 0.001triage_haiku_55_v1 · rescored 77adc97b: 0.001triage_haiku_55_v1 · rescored cd35dc1f: 0.001triage_haiku_55_v1 · repeats ×3: 0.001triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3: 0.001triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3: 0.001triage_haiku_55_v1 · rescored 77adc97b, repeats ×3: 0.001triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3: 0.001triage_gemini_35_flash_lite_v1: 0.0014triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd: 0.0014triage_gemini_35_flash_lite_v1 · rescored 5f2c057c: 0.0014triage_gemini_35_flash_lite_v1 · rescored 77adc97b: 0.0014triage_gemini_35_flash_lite_v1 · rescored cd35dc1f: 0.0014triage_hosted_qwen35_122b_v2: 0.0017triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd: 0.0017triage_hosted_qwen35_122b_v2 · rescored 5f2c057c: 0.0017triage_hosted_qwen35_122b_v1: 0.0033triage_hosted_qwen35_122b_v1 · rescored 5f2c057c: 0.0033triage_hosted_qwen35_122b_v1 · rescored 77adc97b: 0.0033triage_hosted_qwen35_122b_v1 · rescored cd35dc1f: 0.0033triage_baseline_pro_noreason_v1: 0.0049triage_baseline_pro_noreason_v1 · rescored 3d3a6afd: 0.0049triage_baseline_pro_noreason_v1 · rescored 5f2c057c: 0.0049triage_baseline_pro_v1: 0.005triage_baseline_pro_v1 · rescored 3d3a6afd: 0.005triage_baseline_pro_v1 · rescored 5f2c057c: 0.005triage_baseline_pro_v1 · rescored 77adc97b: 0.005triage_baseline_pro_v1 · rescored cd35dc1f: 0.005triage_baseline_pro_v1 · repeats ×3: 0.0051triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3: 0.0051triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3: 0.0051triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3: 0.0051triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3: 0.0051triage_mistral_large_4_v1: 0.0069triage_mistral_large_4_v1 · rescored 5f2c057c: 0.0069triage_mistral_large_4_v1 · rescored 77adc97b: 0.0069triage_mistral_large_4_v1 · rescored cd35dc1f: 0.0069triage_mistral_large_4_v2: 0.0069triage_mistral_large_4_v2 · rescored 3d3a6afd: 0.0069triage_mistral_large_4_v2 · rescored 5f2c057c: 0.0069triage_gpt_61_sol_v1: 0.011triage_gpt_61_sol_v1 · rescored 3d3a6afd: 0.011triage_gpt_61_sol_v1 · rescored 5f2c057c: 0.011triage_gpt_61_sol_v1 · rescored 77adc97b: 0.011triage_gpt_61_sol_v1 · rescored cd35dc1f: 0.011triage_gpt_61_sol_v1 · repeats ×3: 0.011triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3: 0.011triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3: 0.011triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3: 0.011triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3: 0.011triage_hosted_mistral_medium_35_v1: 0.012triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd: 0.012triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c: 0.012triage_hosted_mistral_medium_35_v1 · rescored 77adc97b: 0.012triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f: 0.012triage_kimi_k31_pro_v2: 0.012triage_kimi_k31_pro_v2 · rescored 3d3a6afd: 0.012triage_kimi_k31_pro_v2 · rescored 5f2c057c: 0.012triage_gemini_4_argon_v1: 0.014triage_gemini_4_argon_v1 · rescored 3d3a6afd: 0.014triage_gemini_4_argon_v1 · rescored 5f2c057c: 0.014triage_gemini_4_argon_v1 · rescored 77adc97b: 0.014triage_gemini_4_argon_v1 · rescored cd35dc1f: 0.014triage_kimi_k31_pro_v1: 0.019triage_kimi_k31_pro_v1 · rescored 5f2c057c: 0.019triage_kimi_k31_pro_v1 · rescored 77adc97b: 0.019triage_kimi_k31_pro_v1 · rescored cd35dc1f: 0.019triage_sonnet_55_v1: 0.023triage_sonnet_55_v1 · rescored 3d3a6afd: 0.023triage_sonnet_55_v1 · rescored 5f2c057c: 0.023triage_sonnet_55_v1 · rescored 77adc97b: 0.023triage_sonnet_55_v1 · rescored cd35dc1f: 0.0230 · better0.023

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
triage_local_dgx_gemma4_12b_noreason_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1$0.00000050gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1$0.00000047gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1$0.00000064gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1$0.00000049gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · repeats$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 3d3a6afd$0.00000064gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 3d3a6afd$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c$0.00000050gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c$0.00000047gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 5f2c057c$0.00000064gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c$0.00000049gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 5f2c057c$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 5f2c057c, repeats$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored 77adc97b$0.00000050gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored 77adc97b$0.00000047gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored 77adc97b$0.00000064gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored 77adc97b$0.00000049gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored 77adc97b$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored 77adc97b, repeats$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_12b_noreason_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gemma4_31b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_120b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_gpt_oss_20b_v1 · rescored cd35dc1f$0.00000050gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama31_8b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_llama4_scout_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_ministral_3_14b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_mistral_medium_35_128b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_lightning_30b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_nemotron_35_super_120b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_phi4_14b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen35_122b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen38_27b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_qwen3_30b_a3b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_dgx_vllm_qwen3_30b_a3b_nvfp4_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_12b_noreason_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gemma4_31b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_gpt_oss_20b_v1 · rescored cd35dc1f$0.00000047gold/orrery_triage_v1_labelled gold
triage_local_mac_llama31_8b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_ministral_3_14b_v1 · rescored cd35dc1f$0.00000064gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_nemotron_35_lightning_30b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_phi4_14b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen38_27b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_mac_qwen3_30b_a3b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gemma4_31b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_gpt_oss_20b_v1 · rescored cd35dc1f$0.00000049gold/orrery_triage_v1_labelled gold
triage_local_x3_llama31_8b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_ministral_3_14b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_lightning_30b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_phi4_14b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen38_27b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_qwen3_30b_a3b_v1 · rescored cd35dc1f$0.00000065gold/orrery_triage_v1_labelled gold
triage_local_x3_nemotron_35_bolt_e4b_v1 · rescored cd35dc1f, repeats$0.00000065gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2$0.00035565gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 3d3a6afd$0.00035565gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v2 · rescored 5f2c057c$0.00035565gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2$0.00062765gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 3d3a6afd$0.00062765gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v2 · rescored 5f2c057c$0.00062765gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2$0.00066865gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 3d3a6afd$0.00066865gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v2 · rescored 5f2c057c$0.00066865gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · repeats ×3$0.00068465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c, repeats ×3$0.00068465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b, repeats ×3$0.00068465gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f, repeats ×3$0.00068465gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1$0.00070065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 3d3a6afd$0.00070065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 5f2c057c$0.00070065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored 77adc97b$0.00070065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen3_30b_a3b_v1 · rescored cd35dc1f$0.00070065gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1$0.00071565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 5f2c057c$0.00071565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored 77adc97b$0.00071565gold/orrery_triage_v1_labelled gold
triage_gpt_6_luna_v1 · rescored cd35dc1f$0.00071565gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1$0.00084965gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 5f2c057c$0.00084965gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored 77adc97b$0.00084965gold/orrery_triage_v1_labelled gold
triage_hosted_gpt_oss_120b_v1 · rescored cd35dc1f$0.00084965gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1$0.00086265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 5f2c057c$0.00086265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored 77adc97b$0.00086265gold/orrery_triage_v1_labelled gold
triage_deepseek_41flash_v1 · rescored cd35dc1f$0.00086265gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2$0.00089765gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 3d3a6afd$0.00089765gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v2 · rescored 5f2c057c$0.00089765gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1$0.00090365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 5f2c057c$0.00090365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored 77adc97b$0.00090365gold/orrery_triage_v1_labelled gold
triage_hosted_gemma4_31b_v1 · rescored cd35dc1f$0.00090365gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1$0.00099865gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd$0.00099865gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c$0.00099865gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b$0.00099865gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f$0.00099865gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · repeats ×3$0.00102465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 3d3a6afd, repeats ×3$0.00102465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 5f2c057c, repeats ×3$0.00102465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored 77adc97b, repeats ×3$0.00102465gold/orrery_triage_v1_labelled gold
triage_haiku_55_v1 · rescored cd35dc1f, repeats ×3$0.00102465gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1$0.00136865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 3d3a6afd$0.00136865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 5f2c057c$0.00136865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored 77adc97b$0.00136865gold/orrery_triage_v1_labelled gold
triage_gemini_35_flash_lite_v1 · rescored cd35dc1f$0.00136865gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2$0.00165465gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 3d3a6afd$0.00165465gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v2 · rescored 5f2c057c$0.00165465gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1$0.00335065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 5f2c057c$0.00335065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored 77adc97b$0.00335065gold/orrery_triage_v1_labelled gold
triage_hosted_qwen35_122b_v1 · rescored cd35dc1f$0.00335065gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1$0.00486265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 3d3a6afd$0.00486265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_noreason_v1 · rescored 5f2c057c$0.00486265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1$0.00499265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd$0.00499265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c$0.00499265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b$0.00499265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f$0.00499265gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · repeats ×3$0.00510465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 3d3a6afd, repeats ×3$0.00510465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 5f2c057c, repeats ×3$0.00510465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored 77adc97b, repeats ×3$0.00510465gold/orrery_triage_v1_labelled gold
triage_baseline_pro_v1 · rescored cd35dc1f, repeats ×3$0.00510465gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1$0.00686765gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 5f2c057c$0.00686765gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored 77adc97b$0.00686765gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v1 · rescored cd35dc1f$0.00686765gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2$0.00694865gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 3d3a6afd$0.00694865gold/orrery_triage_v1_labelled gold
triage_mistral_large_4_v2 · rescored 5f2c057c$0.00694865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1$0.01084765gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd$0.01084765gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c$0.01084765gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b$0.01084765gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f$0.01084765gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · repeats ×3$0.01091865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 3d3a6afd, repeats ×3$0.01091865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 5f2c057c, repeats ×3$0.01091865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored 77adc97b, repeats ×3$0.01091865gold/orrery_triage_v1_labelled gold
triage_gpt_61_sol_v1 · rescored cd35dc1f, repeats ×3$0.01091865gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1$0.01153965gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 3d3a6afd$0.01153965gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 5f2c057c$0.01153965gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored 77adc97b$0.01153965gold/orrery_triage_v1_labelled gold
triage_hosted_mistral_medium_35_v1 · rescored cd35dc1f$0.01153965gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2$0.01204165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 3d3a6afd$0.01204165gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v2 · rescored 5f2c057c$0.01204165gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1$0.01361765gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 3d3a6afd$0.01361765gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 5f2c057c$0.01361765gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored 77adc97b$0.01361765gold/orrery_triage_v1_labelled gold
triage_gemini_4_argon_v1 · rescored cd35dc1f$0.01361765gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1$0.01913065gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 5f2c057c$0.01913065gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored 77adc97b$0.01913065gold/orrery_triage_v1_labelled gold
triage_kimi_k31_pro_v1 · rescored cd35dc1f$0.01913065gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1$0.02265965gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 3d3a6afd$0.02265965gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 5f2c057c$0.02265965gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored 77adc97b$0.02265965gold/orrery_triage_v1_labelled gold
triage_sonnet_55_v1 · rescored cd35dc1f$0.02265965gold/orrery_triage_v1_labelled gold

09on scifact_200

gold/scifact_200sf_bge_small_n200_v1: 0sf_bm25_n200_v1: 0sf_e5_base_n200_v1: 0sf_first_k_n200_v1: 0sf_minilm_n200_v1: 0sf_mpnet_n200_v1: 0sf_random_n200_v1: 0sf_deepseek_s_n200_v1: 0.0003sf_gemma_m_n200_v1: 0.00036sf_mistral_s_n200_v1: 0.00043sf_gemma_s_n200_v1: 0.00048sf_llama_s_n200_v1 · repeats: 0.00052sf_glm_s_n200_v1: 0.00071sf_llama_m_n200_v1: 0.00072sf_llama_s_n200_v1: 0.00073sf_glm_s_e5first_n200_v1 · pair: 0.00073sf_glm_s_bm25first_n200_v1 · pair: 0.00074sf_qwen_s_e5first_n200_v1 · pair: 0.00075sf_qwen_s_bm25first_n200_v1 · pair: 0.00078sf_qwen_s_n200_v1: 0.00078sf_qwen_s_n200_v1 · repeats: 0.00078sf_llama_l_n200_v1 · repeats: 0.00085sf_gemma_l_n200_v1: 0.00085sf_llama_l_n200_v1: 0.00087sf_deepseek_m_n200_v1: 0.0012sf_glm_m_n200_v1: 0.00230 · better0.0023

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
sf_bge_small_n200_v1$0.000000200gold/scifact_200 gold
sf_bm25_n200_v1$0.000000200gold/scifact_200 gold
sf_e5_base_n200_v1$0.000000200gold/scifact_200 gold
sf_first_k_n200_v1$0.000000200gold/scifact_200 gold
sf_minilm_n200_v1$0.000000200gold/scifact_200 gold
sf_mpnet_n200_v1$0.000000200gold/scifact_200 gold
sf_random_n200_v1$0.000000200gold/scifact_200 gold
sf_deepseek_s_n200_v1$0.000300200gold/scifact_200 gold
sf_gemma_m_n200_v1$0.000361200gold/scifact_200 gold
sf_mistral_s_n200_v1$0.000426200gold/scifact_200 gold
sf_gemma_s_n200_v1$0.000481200gold/scifact_200 gold
sf_llama_s_n200_v1 · repeats$0.000524200gold/scifact_200 gold
sf_glm_s_n200_v1$0.000711200gold/scifact_200 gold
sf_llama_m_n200_v1$0.000717200gold/scifact_200 gold
sf_llama_s_n200_v1$0.000729200gold/scifact_200 gold
sf_glm_s_e5first_n200_v1 · pair$0.000732200gold/scifact_200 gold
sf_glm_s_bm25first_n200_v1 · pair$0.000741200gold/scifact_200 gold
sf_qwen_s_e5first_n200_v1 · pair$0.000754200gold/scifact_200 gold
sf_qwen_s_bm25first_n200_v1 · pair$0.000775200gold/scifact_200 gold
sf_qwen_s_n200_v1$0.000779200gold/scifact_200 gold
sf_qwen_s_n200_v1 · repeats$0.000779200gold/scifact_200 gold
sf_llama_l_n200_v1 · repeats$0.000846200gold/scifact_200 gold
sf_gemma_l_n200_v1$0.000850200gold/scifact_200 gold
sf_llama_l_n200_v1$0.000865200gold/scifact_200 gold
sf_deepseek_m_n200_v1$0.001190200gold/scifact_200 gold
sf_glm_m_n200_v1$0.002288200gold/scifact_200 gold

10on smoke_v1

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
demo_v1$0.0000005silver/smoke_v1 silver

11on summeval_100

gold/summeval_100se_bartscore_n100_v1: 0se_coverage_n100_v1: 0se_length_n100_v1: 0se_random_n100_pairwise_v1: 0se_random_n100_pick_v1: 0se_random_n100_v1: 0se_rouge1_n100_v1: 0se_rouge2_n100_v1: 0se_rougeL_n100_v1: 0se_unieval_n100_v1: 0se_unievalpub_n100_v1: 0se_gemma_l_n100_pick_v1: 0.00021se_gemma_m_n100_v1: 0.00025se_deepseek_l_n100_v1: 0.00026se_gemma_l_pubtextcw_n100_v1 · pair: 0.00028se_gemma_l_n100_v1: 0.00028se_gemma_s_n100_v1: 0.00028se_qwen_s_n100_pick_v1: 0.00028se_mistral_s_n100_v1: 0.00028se_llama_s_n100_v1: 0.0003se_deepseek_s_n100_v1: 0.00031se_qwen_s_n100_v1: 0.00038se_glm_s_n100_v1: 0.00048se_llama_m_n100_v1: 0.00052se_qwen_s_n100_pairwise_v1: 0.00053se_llama_l_n100_v1: 0.00059se_gemma_l_n100_pairwise_v1: 0.00076se_deepseek_m_n100_v1: 0.00077se_gemma_l_pubtext_n100_v1 · pair: 0.001se_qwen_m_n100_v1: 0.0011se_glm_m_n100_v1: 0.0011se_mistral_m_n100_v1: 0.0012se_mistral_l_n100_v1: 0.0013se_openai_s_n100_v1: 0.0021se_glm_l_n100_v1: 0.0025se_anthropic_s_n100_v1: 0.0032se_qwen37max_n100_v1: 0.0044se_openai_l_n100_v1: 0.0063se_anthropic_m_n100_v1: 0.008se_openai_m_n100_v1: 0.014se_anthropic_l_n100_v1: 0.020 · better0.02

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean cost_usditemsscored against
se_bartscore_n100_v1$0.000000100gold/summeval_100 gold
se_coverage_n100_v1$0.000000100gold/summeval_100 gold
se_length_n100_v1$0.000000100gold/summeval_100 gold
se_random_n100_pairwise_v1$0.000000100gold/summeval_100 gold
se_random_n100_pick_v1$0.000000100gold/summeval_100 gold
se_random_n100_v1$0.000000100gold/summeval_100 gold
se_rouge1_n100_v1$0.000000100gold/summeval_100 gold
se_rouge2_n100_v1$0.000000100gold/summeval_100 gold
se_rougeL_n100_v1$0.000000100gold/summeval_100 gold
se_unieval_n100_v1$0.000000100gold/summeval_100 gold
se_unievalpub_n100_v1$0.000000100gold/summeval_100 gold
se_gemma_l_n100_pick_v1$0.000209100gold/summeval_100 gold
se_gemma_m_n100_v1$0.000252100gold/summeval_100 gold
se_deepseek_l_n100_v1$0.000258100gold/summeval_100 gold
se_gemma_l_pubtextcw_n100_v1 · pair$0.000276100gold/summeval_100 gold
se_gemma_l_n100_v1$0.000278100gold/summeval_100 gold
se_gemma_s_n100_v1$0.000279100gold/summeval_100 gold
se_qwen_s_n100_pick_v1$0.000282100gold/summeval_100 gold
se_mistral_s_n100_v1$0.000283100gold/summeval_100 gold
se_llama_s_n100_v1$0.000297100gold/summeval_100 gold
se_deepseek_s_n100_v1$0.000307100gold/summeval_100 gold
se_qwen_s_n100_v1$0.000385100gold/summeval_100 gold
se_glm_s_n100_v1$0.000481100gold/summeval_100 gold
se_llama_m_n100_v1$0.000517100gold/summeval_100 gold
se_qwen_s_n100_pairwise_v1$0.000531100gold/summeval_100 gold
se_llama_l_n100_v1$0.000594100gold/summeval_100 gold
se_gemma_l_n100_pairwise_v1$0.000764100gold/summeval_100 gold
se_deepseek_m_n100_v1$0.000771100gold/summeval_100 gold
se_gemma_l_pubtext_n100_v1 · pair$0.001013100gold/summeval_100 gold
se_qwen_m_n100_v1$0.001084100gold/summeval_100 gold
se_glm_m_n100_v1$0.001144100gold/summeval_100 gold
se_mistral_m_n100_v1$0.001191100gold/summeval_100 gold
se_mistral_l_n100_v1$0.001257100gold/summeval_100 gold
se_openai_s_n100_v1$0.002124100gold/summeval_100 gold
se_glm_l_n100_v1$0.002451100gold/summeval_100 gold
se_anthropic_s_n100_v1$0.003237100gold/summeval_100 gold
se_qwen37max_n100_v1$0.004436100gold/summeval_100 gold
se_openai_l_n100_v1$0.006350100gold/summeval_100 gold
se_anthropic_m_n100_v1$0.008048100gold/summeval_100 gold
se_openai_m_n100_v1$0.014153100gold/summeval_100 gold
se_anthropic_l_n100_v1$0.020077100gold/summeval_100 gold