arm
triage_local_mac_gemma4_31b_v2
One config on orrery_triage_v1_labelled, scored one way; its repeats averaged per item.
- dataset
- orrery_triage_v1_labelled
- scored against
- gold/orrery_triage_v1_labelled gold
- goal
- Triage decisions on Orrery's error signals
- runs
- 1
- headline
- triage_cost 0.800
- last run
- 2026-11-01 08:13
- re-scored
- the goals read its re-score, runs-rescored/5f2c057cdaa5/runs/orrery_triage_v1_labelled/triage_local_mac_gemma4_31b_v2
01metrics
| metric | mean | items | better |
|---|---|---|---|
| triage_cost | 0.800 | 65 | lower |
| correct | 0.538 | 65 | higher |
| cost_usd | $0.000000 | 65 | lower |
| deferred | 0.000 | 65 | lower |
| escalated | 0.523 | 65 | lower |
| false_alarm | 0.548 | 31 | lower |
| false_dismiss | 0.206 | 34 | lower |
| latency_ms | 25063.1 | 65 | lower |
| model_correct | 0.446 | 65 | higher |
| model_escalated | 0.000 | 65 | lower |
| model_false_alarm | 0.194 | 31 | lower |
| model_false_dismiss | 0.882 | 34 | lower |
| model_triage_cost | 2.400 | 65 | lower |
| n_probes | 1.908 | 65 | higher |
| no_decision | 0.000 | 65 | lower |
| table_miss | 0.000 | 65 | lower |
| tokens_in | 4099.0 | 65 | lower |
| tokens_out | 245.6 | 65 | lower |
| triage_cost_p10 | 1.338 | 65 | lower |
| triage_cost_p3 | 0.585 | 65 | lower |
02cost and speed
- cost per item
- $0.000000 · 4,345 tok
- speed per item
- p50 24,585 · p95 36,311 ms
Per item, over its items: cost as dollars and tokens (a local model is free but still uses tokens), speed as the median and the slow tail (p95). A goal that names this arm weighs them in its decision.
03runs
| run | made | items | cost | how |
|---|---|---|---|---|
triage_local_mac_gemma4_31b_v2_20261101T074610Z | 2026-11-01 08:13 | 65 | $0.000000 | measured |
04config
config triage_local_mac_gemma4_31b_v2 · adapter examples/triage-fleet/adapter.py · deployment mac · credentials are never shown
- call_timeout_s
600- machine
mac- max_tokens
1024- model
gemma4:31b-it-q4_K_M- provider
gateway- reasoning_effort
none- url
http://127.0.0.1:11435/v1/chat/completions- usd_per_mtok_in
0- usd_per_mtok_out
0
05conditions
- data.items_sha256
d428aca49adc8e762dcbbcad6c9a1dba1ac4e4e3315ff26025af975faba1176a- deployment.build
17ba34c06c801ba22f6047bab21ced9429c6d43e0e42ecf339b0f3e5a61e8d97- deployment.machine
mac- deployment.model
gemma4:31b-it-q4_K_M- deployment.quantization
Q4_K_M- deployment.server
ollama 0.40.13- deployment.source
record- fingerprint.version
5- harness.code_sha256
140c7727abcb7dd2630582b7ba69740b6a4d406c969559922c89e48099c3069f- harness.version
0.5.0- library.anthropic
2.3.0- library.numpy
2.5.4- library.openai
3.41.0- library.rouge-score
0.1.2- machine.cpu_count
32- machine.platform
Linux/x86_64- machine.python
3.14.7- machine.release
7.2.5-3-omarchy- reference.sha256
f617ad3bb9bb9f5feb8ffe54e0ef1b9b05dfb316fac13f63a925f9d35fb191cd- scorer.id
examples/triage-fleet/adapter.py- scorer.sha256
3d3a6afdb978f2db2d6c6f732d5842d09c3df0eca24ef724171a162b4b937527
06the same config, scored other ways
scored against silverscored by a judgescored against a label set, not a reference setunmarked rows are scored against gold
| arm | scored against | headline | mean |
|---|---|---|---|
| triage_local_mac_gemma4_31b_v2 · rescored 3d3a6afd | gold/orrery_triage_v1_labelled gold | triage_cost | 0.800 |
| triage_local_mac_gemma4_31b_v2 · rescored 5f2c057c | gold/orrery_triage_v1_labelled gold | triage_cost | 0.800 |
07rounds
- Round r3 of Triage decisions on Orrery's error signals, closed: one of its arms.