metric

`judge_openai_l`

better: higher
kind: quality: measured against a reference or by a judge
decides: no goal
weighed by: no goal
recorded on: cnn_dailymail_200

01what it measures, and how

Judge openai_l's mean score for the output, on the scorer's scale.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"judge_openai_l": {"summary": "…", "details": "…"}}.

02on cnn_dailymail_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean judge_openai_l	items	scored against
cnn_openai_l_n200_v1 · judged/judge-summ-v2	4.957	200	judge `judge-summ-v2`
cnn_openai_m_n200_v1 · judged/judge-summ-v2	4.906	200	judge `judge-summ-v2`
cnn_anthropic_l_n200_v1 · judged/judge-summ-v2	4.865	200	judge `judge-summ-v2`
cnn_deepseek_m_n200_v1 · judged/judge-summ-v2	4.841	200	judge `judge-summ-v2`
cnn_openai_s_n200_v1 · judged/judge-summ-v2	4.804	200	judge `judge-summ-v2`
cnn_deepseek_l_n200_v1 · judged/judge-summ-v2	4.784	200	judge `judge-summ-v2`
cnn_anthropic_m_n200_v1 · judged/judge-summ-v2	4.755	200	judge `judge-summ-v2`
cnn_glm_m_n200_v1 · judged/judge-summ-v2	4.744	200	judge `judge-summ-v2`
cnn_gemma_l_n200_v1 · judged/judge-summ-v2	4.684	200	judge `judge-summ-v2`
cnn_deepseek_s_n200_v1 · judged/judge-summ-v2	4.675	200	judge `judge-summ-v2`
cnn_gemma_m_n200_v1 · judged/judge-summ-v2	4.651	200	judge `judge-summ-v2`
cnn_anthropic_s_n200_v1 · judged/judge-summ-v2	4.646	200	judge `judge-summ-v2`
cnn_llama_l_n200_v1 · judged/judge-summ-v2	4.588	200	judge `judge-summ-v2`
cnn_qwen_l_n200_v1 · judged/judge-summ-v2	4.572	200	judge `judge-summ-v2`
cnn_qwen_m_n200_v1 · judged/judge-summ-v2	4.570	200	judge `judge-summ-v2`
cnn_llama_m_n200_v1 · judged/judge-summ-v2	4.540	200	judge `judge-summ-v2`
cnn_llama_s_n200_v1 · judged/judge-summ-v2	4.514	200	judge `judge-summ-v2`
cnn_qwen_s_n200_v1 · judged/judge-summ-v2	4.503	200	judge `judge-summ-v2`
cnn_glm_s_n200_v1 · judged/judge-summ-v2	4.478	200	judge `judge-summ-v2`
cnn_mistral_s_n200_v1 · judged/judge-summ-v2	4.434	200	judge `judge-summ-v2`
cnn_gemma_s_n200_v1 · judged/judge-summ-v2	4.415	200	judge `judge-summ-v2`
cnn_glm_l_n200_v1 · judged/judge-summ-v2	4.410	200	judge `judge-summ-v2`
cnn_mistral_m_n200_v1 · judged/judge-summ-v2	4.367	200	judge `judge-summ-v2`
cnn_mistral_l_n200_v1 · judged/judge-summ-v2	4.191	200	judge `judge-summ-v2`
cnn_bart_l_n200_v1 · judged/judge-summ-v2	3.929	200	judge `judge-summ-v2`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v2	3.586	200	judge `judge-summ-v2`
cnn_lead3_n200_v1 · judged/judge-summ-v2	3.498	200	judge `judge-summ-v2`
cnn_bart_m_n200_v1 · judged/judge-summ-v2	3.232	200	judge `judge-summ-v2`
cnn_bart_s_n200_v1 · judged/judge-summ-v2	2.935	200	judge `judge-summ-v2`