eval-harness

model

Gemma 4 12B llm

A model record: onboarded once for the project and used by any eval's arms.

weights
open
licence
Apache-2.0
aliases
gemma4:12b
settings
reasoning_effort none at the server's default it reasoned in a loop and never answered (triage, 2026-10-31)
onboarding
onboarded, no arm smoked 2026-10-31
best result per goal
#11 of 55 Triage decisions on Orrery's error signals

Every model

01deployments and their onboarding

x3 · x3 · ollama 0.40.13 · gemma4:12b · onboarding pass 2026-10-31 · 33.6 tokens/s
where
x3
server or device
ollama 0.40.13
build
gemma4:12b
6114515d63c17436a7c0417d82820ac65ad643e2806c5a3c89cb62846436ed0b
Q4_K_M
matches the record
yes
checked
2026-10-31
onboarded on
x3
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11434/v1 · model gemma4:12b—
digest found6114515d63c17436a7c0417d82820ac65ad643e2806c5a3c89cb62846436ed0b—
defaults 9.38 · tokens out 146 · reasoned: yes · finish stop—
chosens 0.69 · tokens out 23 · reasoned: no · finish stop · settings (reasoning effort none)—
tokens per s33.6—
deterministicyes—
json validyes—
dgx · dgx · ollama 0.40.13 · gemma4:12b · onboarding pass 2026-10-31 · 51.3 tokens/s
where
dgx
server or device
ollama 0.40.13
build
gemma4:12b
6114515d63c17436a7c0417d82820ac65ad643e2806c5a3c89cb62846436ed0b
matches the record
yes
checked
2026-10-31
onboarded on
dgx
probemeasurementflag
endpointprovider openai · base url http://100.69.49.126:11434/v1 · model gemma4:12b—
digest found6114515d63c17436a7c0417d82820ac65ad643e2806c5a3c89cb62846436ed0b—
defaults 10.3 · tokens out 270 · reasoned: yes · finish stop—
chosens 0.45 · tokens out 23 · reasoned: no · finish stop · settings (reasoning effort none)—
tokens per s51.3—
deterministicyes—
json validyes—
mac · mac · ollama 0.40.13 · gemma4:12b · onboarding pass 2026-10-31 · 25.1 tokens/s
where
mac
server or device
ollama 0.40.13 context length 32768
build
gemma4:12b
6114515d63c17436a7c0417d82820ac65ad643e2806c5a3c89cb62846436ed0b
Q4_K_M
matches the record
yes
checked
2026-10-31
onboarded on
mac
probemeasurementflag
endpointprovider openai · base url http://127.0.0.1:11435/v1 · model gemma4:12b—
digest found6114515d63c17436a7c0417d82820ac65ad643e2806c5a3c89cb62846436ed0b—
defaults 11.84 · tokens out 146 · reasoned: yes · finish stop—
chosens 0.92 · tokens out 23 · reasoned: no · finish stop · settings (reasoning effort none)—
tokens per s25.1—
deterministicyes—
json validyes—

What it can do, across its deployments: reasons: yes · reasoning off reasoning_effort: none · json mode: yes

02settings history

whenwhat changedwhy
2026-10-31reasoning_effort: (server default) → nonegemma4:12b looped in its reasoning on the triage prompt; with none it answered in 7.7 s
2026-10-31max_tokens, call_timeout_s: 1024, 30 s → (the arms')a reply cap and a call limit are a task's; the triage arms set them (review E6)

03arms

armdeploymentgoalits settingsarm smokeresult
triage_local_dgx_gemma4_12b_noreason_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
dgxTriage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 30nonetriage_cost 0.585
triage_local_mac_gemma4_12b_noreason_v1 · rescored 5f2c057c
orrery_triage_v1_labelled
macTriage decisions on Orrery's error signalsmax_tokens 1024, call_timeout_s 30nonetriage_cost 0.585