metric
judge_qwen_s_unvalidated
- better
- higher
- kind
- quality: measured against a reference or by a judge
- decides
- no goal
- weighed by
- no goal
- recorded on
- cnn_dailymail_200
01what it measures, and how
Judge qwen_s's mean score for the output, on the scorer's scale, not validated against human ratings.
A project explains its own metric at length with METRIC_DESCRIPTIONS = {"judge_qwen_s_unvalidated": {"summary": "…", "details": "…"}}.
02on cnn_dailymail_200
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold
| arm | mean judge_qwen_s_unvalidated | items | scored against |
|---|---|---|---|
| cnn_deepseek_m_n200_v1 · judged/judge-summ-pointwise-v1 | 4.884 | 200 | judge judge-summ-pointwise-v1 |
| cnn_bart_l_n200_v1 · judged/judge-summ-pointwise-v1 | 4.061 | 200 | judge judge-summ-pointwise-v1 |