arm
triage_baseline_pro_v1 baseline
One config on orrery_triage_v1_labelled, scored one way; its repeats averaged per item.
- dataset
- orrery_triage_v1_labelled
- scored against
- gold/orrery_triage_v1_labelled gold
- goal
- Triage decisions on Orrery's error signals
- runs
- 1
- headline
- false_dismiss 0.176
- last run
- 2026-10-30 13:25
- re-scored
- the goals read its re-score, runs-rescored/5f2c057cdaa5/runs/orrery_triage_v1_labelled/triage_baseline_pro_v1
01metrics
| metric | mean | items | better |
|---|---|---|---|
| false_dismiss | 0.176 | 34 | lower |
| correct | 0.569 | 65 | higher |
| cost_usd | $0.004992 | 65 | lower |
| escalated | 0.400 | 65 | lower |
| false_alarm | 0.677 | 31 | lower |
| latency_ms | 22736.3 | 65 | lower |
| model_correct | 0.523 | 65 | higher |
| model_escalated | 0.062 | 65 | lower |
| model_false_alarm | 0.355 | 31 | lower |
| model_false_dismiss | 0.529 | 34 | lower |
| n_probes | 1.538 | 65 | higher |
| table_miss | 0.000 | 65 | lower |
| tokens_in | 3463.1 | 65 | lower |
| tokens_out | 1327.0 | 65 | lower |
02cost and speed
- cost per item
- $0.004992 · 4,790 tok
- speed per item
- p50 19,965 · p95 47,741 ms
Per item, over its items: cost as dollars and tokens (a local model is free but still uses tokens), speed as the median and the slow tail (p95). A goal that names this arm weighs them in its decision.
03runs
| run | made | items | cost | how |
|---|---|---|---|---|
triage_baseline_pro_v1_20261030T132352Z | 2026-10-30 13:25 | 65 | $0.324453 | measured |
04config
config triage_baseline_pro_v1 · adapter examples/triage-fleet/adapter.py · credentials are never shown
- model
eval-fleet-triage-pro- provider
gateway- usd_per_mtok_in
0.816- usd_per_mtok_out
1.632
05deploy sheet
- measured by
triage_baseline_pro_v1_20261030T132352Zon orrery_triage_v1_labelled- against
- gold/orrery_triage_v1_labelled
- model sent
eval-fleet-triage-pro- model record
fleet-triage-pro- deployment
- hosted · homelab LiteLLM (the fleet's gateway) · eval-fleet-triage-pro
- server
- not recorded
- build
- none recorded (a hosted model has no digest)
- settings sent
model: eval-fleet-triage-proprovider: gateway- not sent
- bookkeeping:
usd_per_mtok_in, usd_per_mtok_out
the record setsreasoning_effort: none; this arm did not send it - declared
settings.prompt_sha256: 77f26d87ad9f4d117cd1cbdbd5be9bbb5fcd9666294055c37376fe6938bd1131deployment.fleet_code_sha256: 248e25030011aab82f5798eeb7c74fd03f2d7cfd04866b8b017d3740663727d9deployment.fleet_commit: f07efa9- reported per item
model: eval-fleet-triage-proprompt_sha: c2ece738- adapter
examples/triage-fleet/adapter.pyed10c946cea7- harness
- 0.5.0 44863aeef2ff
No capture here says what production ran (the adapter's production_ran).
Command: evh deploy-sheet --config triage_baseline_pro_v1 [--capture C]
06conditions
- data.items_sha256
d428aca49adc8e762dcbbcad6c9a1dba1ac4e4e3315ff26025af975faba1176a- deployment.model
eval-fleet-triage-pro- deployment.source
adapter- fingerprint.version
5- harness.code_sha256
0ab386969208f46c7448153422d5a5ab07833d963869b69fc216a6cd28feb110- harness.version
0.5.0- library.anthropic
2.3.0- library.numpy
2.5.4- library.openai
3.41.0- library.rouge-score
0.1.2- machine.cpu_count
32- machine.platform
Linux/x86_64- machine.python
3.14.7- machine.release
7.2.5-3-omarchy- reference.sha256
f617ad3bb9bb9f5feb8ffe54e0ef1b9b05dfb316fac13f63a925f9d35fb191cd- scorer.id
examples/triage-fleet/adapter.py- scorer.sha256
ed10c946cea7f30ffd339917d6aeaa7b896b3a7173ae6a06720f8f8c5494e5ae
07the same config, scored other ways
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold
08rounds
- Round r1 of Triage decisions on Orrery's error signals, closed: one of its arms.
- Round r2 of Triage decisions on Orrery's error signals, closed: one of its arms.
- Round r3 of Triage decisions on Orrery's error signals, closed: one of its arms.