eval-harness

arm

db_constant_n200_v1

One config on dbpedia_280, scored one way; its repeats averaged per item.

dataset
dbpedia_280
scored against
gold/dbpedia_280 gold
goal
Encyclopedia topic classification (DBpedia)
runs
1
headline
correct 0.071
last run
2026-09-27 14:12

Inspect itAgainst the baselineWith the 29 arms scored the same way

01metrics

metricmeanitemsbetter
correct0.071280higher
answer_words1.000280neutral
correct_after_repair0.000280neutral
cost_usd$0.000000280lower
fmt_markdown0.000280neutral
fmt_verbose0.000280neutral
latency_ms0.003280lower
parsed1.000280higher

02cost and speed

cost per item
$0.000000
speed per item
p50 0.003 · p95 0.006 ms

Per item, over its items: cost as dollars and tokens (a local model is free but still uses tokens), speed as the median and the slow tail (p95). A goal that names this arm weighs them in its decision.

03runs

runmadeitemscosthow
db_constant_n200_v1_20260927T141254Z2026-09-27 14:12280$0.000000measured

04config

config db_constant_n200_v1 · adapter examples/classification-dbpedia-14/adapter.py · credentials are never shown

family
baseline
label
Company
price_tier
free
provider
constant

05conditions

data.items_sha256
a7419dc6a5876490c3ed1f96eae9398969a2f8b794e744d45ace5790bc0146f2
deployment.model
constant:Company
deployment.source
adapter
fingerprint.version
5
library.openai
3.19.2
machine.cpu_count
12
machine.label
mac-dev
machine.platform
Darwin/x86_64
machine.processor
i386
machine.python
3.11.8
machine.release
24.6.0
reference.sha256
e0564fe65fb011e37717e037a6ad662f44e796b7d7de8e62ed2344ba56fa7a8a
scorer.id
examples/classification-dbpedia-14/adapter.py
scorer.sha256
58b55c964f8bb40746c359621beb25bfd0a071fca079961bdf15be1e7c0a77a8

06rounds