eval-harness

model

gpt-oss 20B llm

A model record: onboarded once for the project and used by any eval's arms.

weights
open
licence
Apache-2.0
aliases
gpt-oss:20b
settings
reasoning_effort low reasoning cannot be turned off: with none it still reasoned on every machine (onboarding, 2026-10-31); low is the least it takes
onboarding
onboarded, no arm smoked 2026-10-31
best result per goal
#26 of 55 Triage decisions on Orrery's error signals

Every model

01deployments and their onboarding

x3 · x3 · ollama 0.40.13 · gpt-oss:20b · onboarding pass 2026-10-31 · 40.9 tokens/s · 1 accepted
where
x3
server or device
ollama 0.40.13
build
gpt-oss:20b
f38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe
MXFP4
matches the record
yes
checked
2026-10-31
onboarded on
x3
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11434/v1 · model gpt-oss:20b—
digest foundf38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe—
defaults 6.97 · tokens out 107 · reasoned: yes · finish stop—
chosens 1.15 · tokens out 47 · reasoned: yes · finish stop · settings (reasoning effort low)—
tokens per s40.9—
deterministicyes—
json validyes—

Accepted: reasoning: it still reasons with the record's settings (it reasons whatever it is asked: with none and with low on every machine (onboarding, 2026-10-31); low is the least)

dgx · dgx · ollama 0.40.13 · gpt-oss:20b · onboarding pass 2026-10-31 · 52.3 tokens/s · 1 accepted
where
dgx
server or device
ollama 0.40.13
build
gpt-oss:20b
f38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe
matches the record
yes
checked
2026-10-31
onboarded on
dgx
probemeasurementflag
endpointprovider openai · base url http://100.69.49.126:11434/v1 · model gpt-oss:20b—
digest foundf38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe—
defaults 8.36 · tokens out 127 · reasoned: yes · finish stop—
chosens 0.9 · tokens out 47 · reasoned: yes · finish stop · settings (reasoning effort low)—
tokens per s52.3—
deterministicyes—
json validyes—

Accepted: reasoning: it still reasons with the record's settings (it reasons whatever it is asked: with none and with low on every machine (onboarding, 2026-10-31); low is the least)

mac · mac · ollama 0.40.13 · gpt-oss:20b · onboarding pass 2026-10-31 · 40.7 tokens/s · 1 accepted
where
mac
server or device
ollama 0.40.13
build
gpt-oss:20b
f38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe
MXFP4
matches the record
yes
checked
2026-10-31
onboarded on
mac
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11435/v1 · model gpt-oss:20b—
digest foundf38aa0c53da5f8c49d08c43a99df24ff53167fe68e24664a7777288e7656fdfe—
defaults 7.17 · tokens out 107 · reasoned: yes · finish stop—
chosens 1.16 · tokens out 47 · reasoned: yes · finish stop · settings (reasoning effort low)—
tokens per s40.7—
deterministicyes—
json validyes—

Accepted: reasoning: it still reasons with the record's settings (it reasons whatever it is asked: with none and with low on every machine (onboarding, 2026-10-31); low is the least)

What it can do, across its deployments: reasons: yes · json mode: yes · reasoning off None

02settings history

whenwhat changedwhy
2026-10-31reasoning_effort: (server default) → noneit reasoned by default on the onboarding probe (2026-10-31); off unless a task needs it (RUNBOOK §16): left at the server's default, one model reasons at length and the next not at all
2026-10-31reasoning_effort: none → lowreasoning cannot be turned off: with none it still reasoned on every machine (onboarding, 2026-10-31); low is the least it takes

03arms

armdeploymentgoalits settingsarm smokeresult
triage_local_dgx_gpt_oss_20b_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
dgxTriage decisions on Orrery's error signalscall_timeout_s 600nonetriage_cost 1.015
triage_local_dgx_gpt_oss_20b_v2 · rescored 5f2c057c
orrery_triage_v1_labelled
dgxTriage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 600check 2026-10-31triage_cost 0.723
triage_local_mac_gpt_oss_20b_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
macTriage decisions on Orrery's error signalscall_timeout_s 600nonetriage_cost 1.092
triage_local_mac_gpt_oss_20b_v2 · rescored 5f2c057c
orrery_triage_v1_labelled
macTriage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 600check 2026-10-31triage_cost 0.708
triage_local_x3_gpt_oss_20b_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
x3Triage decisions on Orrery's error signalscall_timeout_s 600nonetriage_cost 1.092
triage_local_x3_gpt_oss_20b_v2 · rescored 5f2c057c
orrery_triage_v1_labelled
x3Triage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 600check 2026-10-31triage_cost 0.815