eval-harness

model

gpt-oss 120B llm

A model record: onboarded once for the project and used by any eval's arms.

weights
open
licence
Apache-2.0
aliases
gpt-oss:120b eval-gpt-oss-120b
settings
reasoning_effort low reasoning cannot be turned off: with none it still reasoned on every machine, and the hosted route answers "Reasoning is mandatory" (onboarding, 2026-10-31); low is the least it takes
onboarding
vetted 2026-10-31
best result per goal
#16 of 55 Triage decisions on Orrery's error signals

Every model

01deployments and their onboarding

x3 · x3 · ollama 0.40.13 · gpt-oss:120b · onboarding pass 2026-10-31 · 26.8 tokens/s · 1 accepted
where
x3
server or device
ollama 0.40.13
build
gpt-oss:120b
714d86958b6e870a9320e78126ac1ce3477727a1cd9bf7194b8ace0d84c62fd8
MXFP4
matches the record
yes
checked
2026-10-31
onboarded on
x3
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11434/v1 · model gpt-oss:120b—
digest found714d86958b6e870a9320e78126ac1ce3477727a1cd9bf7194b8ace0d84c62fd8—
defaults 18.43 · tokens out 98 · reasoned: yes · finish stop—
chosens 1.83 · tokens out 49 · reasoned: yes · finish stop · settings (reasoning effort low)—
tokens per s26.8—
deterministicyes—
json validyes—

Accepted: reasoning: it still reasons with the record's settings (it reasons whatever it is asked: with none and with low on every machine and the hosted route, which answers "Reasoning is mandatory" (onboarding, 2026-10-31); low is the least)

dgx · dgx · ollama 0.40.13 · gpt-oss:120b · onboarding pass 2026-10-31 · 34.0 tokens/s · 1 accepted
where
dgx
server or device
ollama 0.40.13
build
gpt-oss:120b
714d86958b6e870a9320e78126ac1ce3477727a1cd9bf7194b8ace0d84c62fd8
matches the record
yes
checked
2026-10-31
onboarded on
dgx
probemeasurementflag
endpointprovider openai · base url http://100.69.49.126:11434/v1 · model gpt-oss:120b—
digest found714d86958b6e870a9320e78126ac1ce3477727a1cd9bf7194b8ace0d84c62fd8—
defaults 14.15 · tokens out 98 · reasoned: yes · finish stop—
chosens 1.47 · tokens out 50 · reasoned: yes · finish stop · settings (reasoning effort low)—
tokens per s34.0—
deterministicyes—
json validyes—

Accepted: reasoning: it still reasons with the record's settings (it reasons whatever it is asked: with none and with low on every machine and the hosted route, which answers "Reasoning is mandatory" (onboarding, 2026-10-31); low is the least)

hosted · homelab LiteLLM (the fleet's gateway) · eval-gpt-oss-120b · onboarding pass 2026-10-31 · 28.1 tokens/s · 1 accepted
where
homelab LiteLLM (the fleet's gateway)
build
eval-gpt-oss-120b
matches the record
yes
checked
2026-10-31
onboarded on
homelab LiteLLM (the fleet's gateway)
probemeasurementflag
endpointprovider litellm · base url http://homelab:4001/v1 · model eval-gpt-oss-120b—
digest foundNone—
defaults 2.28 · tokens out 83 · reasoned: yes · finish stop—
chosens 1.35 · tokens out 38 · reasoned: yes · finish stop · settings (reasoning effort low)—
tokens per s28.1—
deterministicno—
json validyes—

Accepted: reasoning: it still reasons with the record's settings (it reasons whatever it is asked: with none and with low on every machine and the hosted route, which answers "Reasoning is mandatory" (onboarding, 2026-10-31); low is the least)

What it can do, across its deployments: reasons: yes · json mode: yes · reasoning off None

02settings history

whenwhat changedwhy
2026-10-31reasoning_effort: (server default) → noneit reasoned by default on the onboarding probe (2026-10-31); off unless a task needs it (RUNBOOK §16): left at the server's default, one model reasons at length and the next not at all
2026-10-31reasoning_effort: none → lowreasoning cannot be turned off: with none it still reasoned on every machine, and the hosted route answers "Reasoning is mandatory" (onboarding, 2026-10-31); low is the least it takes