metric

`judge_anthropic_l`

better: higher
kind: quality: measured against a reference or by a judge
decides: no goal
weighed by: no goal
recorded on: cnn_dailymail_200

01what it measures, and how

Judge anthropic_l's mean score for the output, on the scorer's scale.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"judge_anthropic_l": {"summary": "…", "details": "…"}}.

02on cnn_dailymail_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean judge_anthropic_l	items	scored against
cnn_anthropic_l_n200_v1 · judged/judge-summ-v2	4.911	200	judge `judge-summ-v2`
cnn_deepseek_m_n200_v1 · judged/judge-summ-v2	4.866	200	judge `judge-summ-v2`
cnn_anthropic_m_n200_v1 · judged/judge-summ-v2	4.862	200	judge `judge-summ-v2`
cnn_anthropic_s_n200_v1 · judged/judge-summ-v2	4.810	200	judge `judge-summ-v2`
cnn_deepseek_l_n200_v1 · judged/judge-summ-v2	4.741	200	judge `judge-summ-v2`
cnn_openai_l_n200_v1 · judged/judge-summ-v2	4.736	200	judge `judge-summ-v2`
cnn_openai_m_n200_v1 · judged/judge-summ-v2	4.716	200	judge `judge-summ-v2`
cnn_glm_m_n200_v1 · judged/judge-summ-v2	4.692	200	judge `judge-summ-v2`
cnn_openai_s_n200_v1 · judged/judge-summ-v2	4.691	200	judge `judge-summ-v2`
cnn_qwen_m_n200_v1 · judged/judge-summ-v2	4.630	200	judge `judge-summ-v2`
cnn_deepseek_s_n200_v1 · judged/judge-summ-v2	4.612	200	judge `judge-summ-v2`
cnn_qwen_l_n200_v1 · judged/judge-summ-v2	4.574	200	judge `judge-summ-v2`
cnn_gemma_l_n200_v1 · judged/judge-summ-v2	4.522	200	judge `judge-summ-v2`
cnn_gemma_m_n200_v1 · judged/judge-summ-v2	4.519	200	judge `judge-summ-v2`
cnn_llama_l_n200_v1 · judged/judge-summ-v2	4.471	200	judge `judge-summ-v2`
cnn_gemma_s_n200_v1 · judged/judge-summ-v2	4.442	200	judge `judge-summ-v2`
cnn_llama_m_n200_v1 · judged/judge-summ-v2	4.436	200	judge `judge-summ-v2`
cnn_llama_s_n200_v1 · judged/judge-summ-v2	4.425	200	judge `judge-summ-v2`
cnn_qwen_s_n200_v1 · judged/judge-summ-v2	4.419	200	judge `judge-summ-v2`
cnn_mistral_s_n200_v1 · judged/judge-summ-v2	4.383	200	judge `judge-summ-v2`
cnn_glm_s_n200_v1 · judged/judge-summ-v2	4.362	200	judge `judge-summ-v2`
cnn_glm_l_n200_v1 · judged/judge-summ-v2	4.357	200	judge `judge-summ-v2`
cnn_mistral_m_n200_v1 · judged/judge-summ-v2	4.250	200	judge `judge-summ-v2`
cnn_mistral_l_n200_v1 · judged/judge-summ-v2	3.884	200	judge `judge-summ-v2`
cnn_bart_l_n200_v1 · judged/judge-summ-v2	3.712	200	judge `judge-summ-v2`
cnn_bart_m_n200_v1 · judged/judge-summ-v2	3.237	200	judge `judge-summ-v2`
cnn_lead3_n200_v1 · judged/judge-summ-v2	3.180	200	judge `judge-summ-v2`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2	3.146	200	judge `judge-summ-v2`
cnn_bart_s_n200_v1 · judged/judge-summ-v2	2.958	200	judge `judge-summ-v2`