metric
rho
- better
- higher
- kind
- quality: measured against a reference or by a judge
- decides
- no goal
- weighed by
- no goal
- recorded on
- summeval_100
01what it measures, and how
Spearman's rho between the judge's scores and the experts' ratings of an article's 16 summaries, averaged over the four dimensions: kept for comparison with the literature.
A project explains its own metric at length with METRIC_DESCRIPTIONS = {"rho": {"summary": "…", "details": "…"}}.
02on summeval_100
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold