metric
output_words
- better
- neutral: a description
- kind
- descriptive: says what the output was, never whether it was good
- decides
- no goal
- weighed by
- no goal
- recorded on
- smoke_v1
01what it measures, and how
How many words the output has: a description of the output, not a quality.
The number of whitespace-separated words in the output. It says how long the output was, never whether it was good: ranking by it rewards verbosity, and the leaderboard says so when a table is sorted by it.
02on smoke_v1
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold
| arm | mean output_words | items | scored against |
|---|---|---|---|
| demo_v1 | 34.8 | 5 | silver/smoke_v1 silver |