metric

`untyped_fn`

better: neutral: a description
kind: descriptive: says what the output was, never whether it was good
decides: no goal
weighed by: no goal
recorded on: few_nerd_280

01what it measures, and how

False negatives on texts alone, types ignored.

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"untyped_fn": {"summary": "…", "details": "…"}}.

02on few_nerd_280

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

arm	mean untyped_fn	items	scored against
fn_nothing_n200_v1	2.736	280	gold/few_nerd_280 gold
fn_gliner_n200_v1	0.907	280	gold/few_nerd_280 gold
fn_glm_l_n200_v1	0.761	280	gold/few_nerd_280 gold
fn_glm_s_n200_v1	0.696	280	gold/few_nerd_280 gold
fn_capitalized_n200_v1	0.629	280	gold/few_nerd_280 gold
fn_qwen_l_n200_v1	0.621	280	gold/few_nerd_280 gold
fn_mistral_m_n200_v1	0.589	280	gold/few_nerd_280 gold
fn_llama_s_n200_v1	0.586	280	gold/few_nerd_280 gold
fn_anthropic_m_n200_v1	0.557	280	gold/few_nerd_280 gold
fn_llama_m_n200_v1	0.554	280	gold/few_nerd_280 gold
fn_mistral_l_n200_v1	0.546	280	gold/few_nerd_280 gold
fn_mistral_s_n200_v1	0.546	280	gold/few_nerd_280 gold
fn_openai_s_n200_v1	0.525	280	gold/few_nerd_280 gold
fn_anthropic_s_n200_v1	0.514	280	gold/few_nerd_280 gold
fn_llama_l_n200_v1	0.504	280	gold/few_nerd_280 gold
fn_qwen_s_n200_v1	0.493	280	gold/few_nerd_280 gold
fn_deepseek_s_n200_v1	0.486	280	gold/few_nerd_280 gold
fn_glm_m_n200_v1	0.486	280	gold/few_nerd_280 gold
fn_gemma_m_n200_v1	0.482	280	gold/few_nerd_280 gold
fn_openai_m_n200_v1	0.471	280	gold/few_nerd_280 gold
fn_gemma_l_n200_v1	0.468	280	gold/few_nerd_280 gold
fn_qwen_m_n200_v1	0.468	280	gold/few_nerd_280 gold
fn_deepseek_m_n200_v1	0.457	280	gold/few_nerd_280 gold
fn_deepseek_l_n200_v1	0.454	280	gold/few_nerd_280 gold
fn_openai_l_n200_v1	0.446	280	gold/few_nerd_280 gold
fn_anthropic_l_n200_v1	0.436	280	gold/few_nerd_280 gold
fn_gemma_s_n200_v1	0.436	280	gold/few_nerd_280 gold
fn_span_marker_n200_v1	0.361	280	gold/few_nerd_280 gold