metric
ndcg_10
- better
- higher
- kind
- quality: measured against a reference or by a judge
- decides
- Evidence retrieval for scientific claims (SciFact)
- weighed by
- no goal
- recorded on
- scifact_200
01what it measures, and how
nDCG@10: how high the relevant documents sit in the first 10, against the best possible order.
Gain 2^grade − 1 per relevant document (grade above 0), discounted by log2 of rank + 1, divided by the ideal ordering's DCG. Ids are normalised and duplicates collapsed (first occurrence wins); an id that is not a corpus document keeps its slot and earns nothing. An unreadable reply scores 0.
02on scifact_200
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold