metric

`untyped_f1`

better: higher
kind: quality: measured against a reference or by a judge
decides: no goal
weighed by: no goal
recorded on: few_nerd_280

01what it measures, and how

F1 on the entity texts alone, types ignored: did the arm find the entity at all.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"untyped_f1": {"summary": "…", "details": "…"}}.

02on few_nerd_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean untyped_f1	items	scored against
fn_span_marker_n200_v1	0.837	280	gold/few_nerd_280 gold
fn_anthropic_l_n200_v1	0.782	280	gold/few_nerd_280 gold
fn_openai_m_n200_v1	0.778	280	gold/few_nerd_280 gold
fn_gemma_m_n200_v1	0.775	280	gold/few_nerd_280 gold
fn_openai_l_n200_v1	0.769	280	gold/few_nerd_280 gold
fn_gemma_l_n200_v1	0.769	280	gold/few_nerd_280 gold
fn_anthropic_m_n200_v1	0.767	280	gold/few_nerd_280 gold
fn_qwen_s_n200_v1	0.765	280	gold/few_nerd_280 gold
fn_qwen_m_n200_v1	0.753	280	gold/few_nerd_280 gold
fn_qwen_l_n200_v1	0.728	280	gold/few_nerd_280 gold
fn_anthropic_s_n200_v1	0.728	280	gold/few_nerd_280 gold
fn_deepseek_m_n200_v1	0.727	280	gold/few_nerd_280 gold
fn_glm_m_n200_v1	0.715	280	gold/few_nerd_280 gold
fn_llama_l_n200_v1	0.715	280	gold/few_nerd_280 gold
fn_deepseek_s_n200_v1	0.700	280	gold/few_nerd_280 gold
fn_mistral_l_n200_v1	0.695	280	gold/few_nerd_280 gold
fn_deepseek_l_n200_v1	0.694	280	gold/few_nerd_280 gold
fn_mistral_m_n200_v1	0.687	280	gold/few_nerd_280 gold
fn_glm_s_n200_v1	0.686	280	gold/few_nerd_280 gold
fn_openai_s_n200_v1	0.685	280	gold/few_nerd_280 gold
fn_llama_s_n200_v1	0.681	280	gold/few_nerd_280 gold
fn_llama_m_n200_v1	0.677	280	gold/few_nerd_280 gold
fn_gemma_s_n200_v1	0.676	280	gold/few_nerd_280 gold
fn_glm_l_n200_v1	0.653	280	gold/few_nerd_280 gold
fn_mistral_s_n200_v1	0.645	280	gold/few_nerd_280 gold
fn_capitalized_n200_v1	0.619	280	gold/few_nerd_280 gold
fn_gliner_n200_v1	0.552	280	gold/few_nerd_280 gold
fn_nothing_n200_v1	0.125	280	gold/few_nerd_280 gold