eval-harness

model

Qwen 3.8 27B llm

A model record: onboarded once for the project and used by any eval's arms.

weights
open
licence
Apache-2.0
aliases
qwen3.8:27b
settings
reasoning_effort none it reasoned by default on the onboarding probe (2026-10-31); off unless a task needs it (RUNBOOK §16): left at the server's default, one model reasons at length and the next not at all
onboarding
vetted 2026-10-31
best result per goal
#50 of 55 Triage decisions on Orrery's error signals

Every model

01deployments and their onboarding

x3 · x3 · ollama 0.40.13 · qwen3.8:27b · onboarding pass 2026-10-31 · 22.8 tokens/s
where
x3
server or device
ollama 0.40.13
build
qwen3.8:27b
aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813
Q4_K_M
matches the record
yes
checked
2026-10-31
onboarded on
x3
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11434/v1 · model qwen3.8:27b—
digest foundaaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813—
defaults 8.66 · tokens out 79 · reasoned: yes · finish stop—
chosens 1.01 · tokens out 23 · reasoned: no · finish stop · settings (reasoning effort none)—
tokens per s22.8—
deterministicyes—
json validyes—
dgx · dgx · ollama 0.40.13 · qwen3.8:27b · onboarding pass 2026-10-31 · 23.7 tokens/s
where
dgx
server or device
ollama 0.40.13
build
qwen3.8:27b
aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813
matches the record
yes
checked
2026-10-31
onboarded on
dgx
probemeasurementflag
endpointprovider openai · base url http://100.69.49.126:11434/v1 · model qwen3.8:27b—
digest foundaaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813—
defaults 8.18 · tokens out 67 · reasoned: yes · finish stop—
chosens 0.97 · tokens out 23 · reasoned: no · finish stop · settings (reasoning effort none)—
tokens per s23.7—
deterministicyes—
json validyes—
mac · mac · ollama 0.40.13 · qwen3.8:27b · onboarding pass 2026-10-31 · 20.2 tokens/s
where
mac
server or device
ollama 0.40.13
build
qwen3.8:27b
aaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813
Q4_K_M
matches the record
yes
checked
2026-10-31
onboarded on
mac
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11435/v1 · model qwen3.8:27b—
digest foundaaee06c39dcf2437cde036998d960e1fc1494b8191be7cc9657d01e509097813—
defaults 16.75 · tokens out 73 · reasoned: yes · finish stop—
chosens 1.14 · tokens out 23 · reasoned: no · finish stop · settings (reasoning effort none)—
tokens per s20.2—
deterministicyes—
json validyes—

What it can do, across its deployments: reasons: yes · json mode: yes · reasoning off reasoning_effort: none

02settings history

whenwhat changedwhy
2026-10-31reasoning_effort: (server default) → noneit reasoned by default on the onboarding probe (2026-10-31); off unless a task needs it (RUNBOOK §16): left at the server's default, one model reasons at length and the next not at all

03arms

armdeploymentgoalits settingsarm smokeresult
triage_local_dgx_qwen38_27b_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
dgxTriage decisions on Orrery's error signalscall_timeout_s 600nonetriage_cost 0.831
triage_local_dgx_qwen38_27b_v2 · rescored 5f2c057c
orrery_triage_v1_labelled
dgxTriage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 600pass 2026-10-31triage_cost 1.138
triage_local_mac_qwen38_27b_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
macTriage decisions on Orrery's error signalscall_timeout_s 600nonetriage_cost 1.046
triage_local_mac_qwen38_27b_v2 · rescored 5f2c057c
orrery_triage_v1_labelled
macTriage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 600pass 2026-10-31triage_cost 1.138
triage_local_x3_qwen38_27b_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
x3Triage decisions on Orrery's error signalscall_timeout_s 600nonetriage_cost 0.908
triage_local_x3_qwen38_27b_v2 · rescored 5f2c057c
orrery_triage_v1_labelled
x3Triage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 600check 2026-10-31triage_cost 1.292