f1

01what it measures, and how

Typed entity F1 for the item: an entity counts when its text and its type both match a gold one.

Texts are normalised (casefolded, accents and punctuation stripped, a leading article dropped, no stemming) and matched one to one, so one prediction can satisfy one gold entity at most. An item with no gold and no prediction scores 1; an unreadable reply scores 0 and counts every gold entity as missed. The mean over items is a macro F1, not the corpus micro F1, which tp, fp and fn let a report pool.

02on few_nerd_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold