metric

`judge_gemma_l`

better: higher
kind: quality: measured against a reference or by a judge
decides: no goal
weighed by: no goal
recorded on: cnn_dailymail_200

01what it measures, and how

Judge gemma_l's mean score for the output, on the scorer's scale.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"judge_gemma_l": {"summary": "…", "details": "…"}}.

02on cnn_dailymail_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean judge_gemma_l	items	scored against
cnn_anthropic_l_n200_v1 · judged/judge-summ-v1	4.995	197	judge `judge-summ-v1`
cnn_anthropic_m_n200_v1 · judged/judge-summ-v1	4.985	197	judge `judge-summ-v1`
cnn_anthropic_s_n200_v1 · judged/judge-summ-v1	4.976	197	judge `judge-summ-v1`
cnn_deepseek_m_n200_v1 · judged/judge-summ-v1	4.967	197	judge `judge-summ-v1`
cnn_deepseek_l_n200_v1 · judged/judge-summ-v1	4.937	197	judge `judge-summ-v1`
cnn_glm_m_n200_v1 · judged/judge-summ-v1	4.937	197	judge `judge-summ-v1`
cnn_qwen_m_n200_v1 · judged/judge-summ-v1	4.923	197	judge `judge-summ-v1`
cnn_deepseek_s_n200_v1 · judged/judge-summ-v1	4.906	197	judge `judge-summ-v1`
cnn_mistral_s_n200_v1 · judged/judge-summ-v1	4.902	197	judge `judge-summ-v1`
cnn_gemma_s_n200_v1 · judged/judge-summ-v1	4.901	197	judge `judge-summ-v1`
cnn_openai_s_n200_v1 · judged/judge-summ-v1	4.897	197	judge `judge-summ-v1`
cnn_mistral_l_n200_v1 · judged/judge-summ-v1	4.887	197	judge `judge-summ-v1`
cnn_openai_m_n200_v1 · judged/judge-summ-v1	4.877	197	judge `judge-summ-v1`
cnn_openai_l_n200_v1 · judged/judge-summ-v1	4.869	197	judge `judge-summ-v1`
cnn_gemma_l_n200_v1 · judged/judge-summ-v1	4.855	197	judge `judge-summ-v1`
cnn_qwen_l_n200_v1 · judged/judge-summ-v1	4.850	197	judge `judge-summ-v1`
cnn_llama_l_n200_v1 · judged/judge-summ-v1	4.840	197	judge `judge-summ-v1`
cnn_mistral_m_n200_v1 · judged/judge-summ-v1	4.840	197	judge `judge-summ-v1`
cnn_llama_s_n200_v1 · judged/judge-summ-v1	4.832	197	judge `judge-summ-v1`
cnn_llama_m_n200_v1 · judged/judge-summ-v1	4.829	197	judge `judge-summ-v1`
cnn_gemma_m_n200_v1 · judged/judge-summ-v1	4.827	197	judge `judge-summ-v1`
cnn_qwen_s_n200_v1 · judged/judge-summ-v1	4.730	197	judge `judge-summ-v1`
cnn_glm_s_n200_v1 · judged/judge-summ-v1	4.722	197	judge `judge-summ-v1`
cnn_glm_l_n200_v1 · judged/judge-summ-v1	4.678	197	judge `judge-summ-v1`
cnn_bart_l_n200_v1 · judged/judge-summ-v1	4.129	197	judge `judge-summ-v1`
cnn_bart_m_n200_v1 · judged/judge-summ-v1	3.957	197	judge `judge-summ-v1`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-v1	3.887	197	judge `judge-summ-v1`
cnn_lead3_n200_v1 · judged/judge-summ-v1	3.721	197	judge `judge-summ-v1`
cnn_bart_s_n200_v1 · judged/judge-summ-v1	3.542	197	judge `judge-summ-v1`
cnn_anthropic_l_n200_v1 · judged/judge-summ-pick-v1	0.385	198	judge `judge-summ-pick-v1`
cnn_anthropic_s_n200_v1 · judged/judge-summ-pick-v1	0.241	198	judge `judge-summ-pick-v1`
cnn_anthropic_m_n200_v1 · judged/judge-summ-pick-v1	0.146	198	judge `judge-summ-pick-v1`
cnn_deepseek_m_n200_v1 · judged/judge-summ-pick-v1	0.091	198	judge `judge-summ-pick-v1`
cnn_qwen_l_n200_v1 · judged/judge-summ-pick-v1	0.048	198	judge `judge-summ-pick-v1`
cnn_qwen_m_n200_v1 · judged/judge-summ-pick-v1	0.015	198	judge `judge-summ-pick-v1`
cnn_glm_l_n200_v1 · judged/judge-summ-pick-v1	0.013	198	judge `judge-summ-pick-v1`
cnn_glm_m_n200_v1 · judged/judge-summ-pick-v1	0.011	198	judge `judge-summ-pick-v1`
cnn_mistral_s_n200_v1 · judged/judge-summ-pick-v1	0.008	198	judge `judge-summ-pick-v1`
cnn_deepseek_l_n200_v1 · judged/judge-summ-pick-v1	0.007	198	judge `judge-summ-pick-v1`
cnn_deepseek_s_n200_v1 · judged/judge-summ-pick-v1	0.007	198	judge `judge-summ-pick-v1`
cnn_gemma_m_n200_v1 · judged/judge-summ-pick-v1	0.006	198	judge `judge-summ-pick-v1`
cnn_openai_s_n200_v1 · judged/judge-summ-pick-v1	0.005	198	judge `judge-summ-pick-v1`
cnn_openai_l_n200_v1 · judged/judge-summ-pick-v1	0.004	198	judge `judge-summ-pick-v1`
cnn_gemma_s_n200_v1 · judged/judge-summ-pick-v1	0.003	198	judge `judge-summ-pick-v1`
cnn_llama_s_n200_v1 · judged/judge-summ-pick-v1	0.003	198	judge `judge-summ-pick-v1`
cnn_qwen_s_n200_v1 · judged/judge-summ-pick-v1	0.002	198	judge `judge-summ-pick-v1`
cnn_glm_s_n200_v1 · judged/judge-summ-pick-v1	0.002	198	judge `judge-summ-pick-v1`
cnn_openai_m_n200_v1 · judged/judge-summ-pick-v1	0.002	198	judge `judge-summ-pick-v1`
cnn_gemma_l_n200_v1 · judged/judge-summ-pick-v1	0.001	198	judge `judge-summ-pick-v1`
cnn_llama_m_n200_v1 · judged/judge-summ-pick-v1	0.001	198	judge `judge-summ-pick-v1`
cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1	0.001	198	judge `judge-summ-pick-v1`
cnn_bart_l_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`
cnn_bart_l_xsum_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`
cnn_bart_m_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`
cnn_bart_s_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`
cnn_lead3_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`
cnn_mistral_l_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`
cnn_mistral_m_n200_v1 · judged/judge-summ-pick-v1	0.000	198	judge `judge-summ-pick-v1`