truncated

01what it measures, and how

1 when the provider cut the reply off at the token ceiling (finish_reason "length", and in most adapters a completion that reached 98% of max_tokens), else 0. An arm with truncated above 0 has scores that partly describe the ceiling, not the model.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"truncated": {"summary": "…", "details": "…"}}.

02on ag_news_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

03on cnn_dailymail_200

Not drawn: cnn_anthropic_l_n200_v1 · silver, scored against the reference it wrote (its row is in the table).

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

04on dbpedia_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

05on few_nerd_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

06on scifact_200

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

07on summeval_100

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold