dataset
orrery_triage_smoke
2 items · 0 arms. Headline: each arm's own primary metric, mean over its items; arms with different primary metrics are not ranked against each other.
01arms
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold
| arm | metric | mean | items | cost per item | speed per item | runs | scored against |
|---|
02what's inside
2 items, json, 167–195 words (median 195, middle half 167–195).
the gold
gold/orrery_triage_smoke: text, 1–1 words, median 1
| item | words | gold | open |
|---|---|---|---|
orrery-ORRER | 195 | 1 words | — |
orrery-ORRER | 167 | 1 words | — |
Each row is the profile evh dataset-items --dataset-id orrery_triage_smoke prints; inspect opens the item in the inspector with the dataset's largest group of comparable arms, and its text where the inspector runs on a machine that has it.
03reference sets
- gold/orrery_triage_smoke gold · promoted from person
gold_orrery_triage_smoke
04labels
| set | made on | coverage | guideline | made | promoted |
|---|---|---|---|---|---|
person gold_orrery_triage_smoke | this one | 2 of 2 | none | person · owner 2026-11-01 | gold/orrery_triage_smoke gold |
person gold_orrery_triage_v1_labelled | orrery_triage_v1_labelled | 2 of 2 | none | person · owner 2026-11-01 | gold/orrery_triage_v1_labelled gold |
05item fields
Declared in the field set triage. Values are summarised only for fields marked public; the inspector shows each item's, filters a comparison by them, and breaks it down by a choice or yes/no field.
| field | type | filled | description | values |
|---|---|---|---|---|
Evidenceevidence_level | choice | 2 of 2 | exact: the probe answers the model saw; the rest rebuilt for the decision's time, or unavailable. | exact 2 |
Decided bydecision_kind | choice | 0 of 2 | model: the model decided; triager_failure: it never did (the 2026-08 authentication outage). | — |
Production decidedfleet_disposition | choice | 2 of 2 | What the fleet decided in production. | escalate 1 · cleanup 1 |
Production modelfleet_model | text | 0 of 2 | The gateway alias production used. | — |
Stagestage | text | 0 of 2 | The fleet's rollout stage for the decision (propose, shadow; empty for a test signal). | — |
Fingerprintfingerprint | text | 2 of 2 | The signal's fingerprint; decisions on one fingerprint are related. | 2 distinct |
Decided ondecision_date | date | 0 of 2 | The day the fleet decided. | — |
Owner-labelledlabelled | choice | 2 of 2 | yes: the owner's label exists for this decision (verdicts.jsonl, source owner-*). | yes 2 |