eval-harness

metric

tp

better
neutral: a description
kind
descriptive: says what the output was, never whether it was good
decides
no goal
weighed by
no goal
recorded on
few_nerd_280

01what it measures, and how

Typed true positives: predicted entities matched to a gold one (text and type).

A project explains its own metric at length with METRIC_DESCRIPTIONS = {"tp": {"summary": "…", "details": "…"}}.

02on few_nerd_280

gold/few_nerd_280fn_span_marker_n200_v1: 2.196fn_openai_l_n200_v1: 2.018fn_anthropic_l_n200_v1: 2.007fn_openai_m_n200_v1: 1.989fn_qwen_m_n200_v1: 1.964fn_qwen_s_n200_v1: 1.964fn_openai_s_n200_v1: 1.946fn_deepseek_l_n200_v1: 1.943fn_gemma_m_n200_v1: 1.943fn_gemma_l_n200_v1: 1.9fn_mistral_l_n200_v1: 1.882fn_anthropic_m_n200_v1: 1.875fn_llama_m_n200_v1: 1.871fn_deepseek_m_n200_v1: 1.861fn_gemma_s_n200_v1: 1.861fn_llama_l_n200_v1: 1.839fn_mistral_m_n200_v1: 1.839fn_deepseek_s_n200_v1: 1.832fn_glm_m_n200_v1: 1.8fn_qwen_l_n200_v1: 1.786fn_llama_s_n200_v1: 1.764fn_mistral_s_n200_v1: 1.746fn_anthropic_s_n200_v1: 1.729fn_glm_l_n200_v1: 1.646fn_glm_s_n200_v1: 1.582fn_gliner_n200_v1: 1.546fn_capitalized_n200_v1: 0.618fn_nothing_n200_v1: 00better · 2.196

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold

armmean tpitemsscored against
fn_span_marker_n200_v12.196280gold/few_nerd_280 gold
fn_openai_l_n200_v12.018280gold/few_nerd_280 gold
fn_anthropic_l_n200_v12.007280gold/few_nerd_280 gold
fn_openai_m_n200_v11.989280gold/few_nerd_280 gold
fn_qwen_m_n200_v11.964280gold/few_nerd_280 gold
fn_qwen_s_n200_v11.964280gold/few_nerd_280 gold
fn_openai_s_n200_v11.946280gold/few_nerd_280 gold
fn_deepseek_l_n200_v11.943280gold/few_nerd_280 gold
fn_gemma_m_n200_v11.943280gold/few_nerd_280 gold
fn_gemma_l_n200_v11.900280gold/few_nerd_280 gold
fn_mistral_l_n200_v11.882280gold/few_nerd_280 gold
fn_anthropic_m_n200_v11.875280gold/few_nerd_280 gold
fn_llama_m_n200_v11.871280gold/few_nerd_280 gold
fn_deepseek_m_n200_v11.861280gold/few_nerd_280 gold
fn_gemma_s_n200_v11.861280gold/few_nerd_280 gold
fn_llama_l_n200_v11.839280gold/few_nerd_280 gold
fn_mistral_m_n200_v11.839280gold/few_nerd_280 gold
fn_deepseek_s_n200_v11.832280gold/few_nerd_280 gold
fn_glm_m_n200_v11.800280gold/few_nerd_280 gold
fn_qwen_l_n200_v11.786280gold/few_nerd_280 gold
fn_llama_s_n200_v11.764280gold/few_nerd_280 gold
fn_mistral_s_n200_v11.746280gold/few_nerd_280 gold
fn_anthropic_s_n200_v11.729280gold/few_nerd_280 gold
fn_glm_l_n200_v11.646280gold/few_nerd_280 gold
fn_glm_s_n200_v11.582280gold/few_nerd_280 gold
fn_gliner_n200_v11.546280gold/few_nerd_280 gold
fn_capitalized_n200_v10.618280gold/few_nerd_280 gold
fn_nothing_n200_v10.000280gold/few_nerd_280 gold