correct_after_repair

01what it measures, and how

1 when the answer is correct only because the parser read it: the label was right but the reply was not exactly the label. A large share means the accuracy partly measures the parser.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"correct_after_repair": {"summary": "…", "details": "…"}}.

02on ag_news_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

03on dbpedia_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold