arm
se_random_n100_pairwise_v1
One config on summeval_100, scored one way; its repeats averaged per item.
- dataset
- summeval_100
- scored against
- gold/summeval_100 gold
- runs
- 1
- headline
- pair_agreement 0.473
- last run
- 2026-11-02 14:14
01metrics
| metric | mean | items | better |
|---|---|---|---|
| pair_agreement | 0.473 | 100 | higher |
| cost_usd | $0.000000 | 100 | lower |
| latency_ms | 0.073 | 100 | lower |
| n_pairs | 7.490 | 100 | neutral |
| parsed | 1.000 | 100 | neutral |
02cost and speed
- cost per item
- $0.000000
- speed per item
- p50 0.064 · p95 0.12 ms
Per item, over its items: cost as dollars and tokens (a local model is free but still uses tokens), speed as the median and the slow tail (p95). A goal that names this arm weighs them in its decision.
03runs
| run | made | items | cost | how |
|---|---|---|---|---|
se_random_n100_pairwise_v1_20261102T141457Z | 2026-11-02 14:14 | 100 | $0.000000 | measured |
04config
config se_random_n100_pairwise_v1 · adapter examples/summeval-judging/adapter.py · credentials are never shown
- family
baseline- mode
pairwise- price_tier
free- provider
random- seed
20261015
05conditions
- data.items_sha256
b59999b281b2cbe87c57e4af34cc475c01ba1d531c7b77cb93a136665119c613- deployment.model
random- deployment.source
params- fingerprint.version
5- harness.code_sha256
b009ef6f326416a083541a1c55f6bac169410d4cfcc49b9266dcf9df1d336bb5- harness.version
0.5.0- library.anthropic
2.3.0- library.numpy
2.5.4- library.openai
3.41.0- library.rouge-score
0.1.2- machine.cpu_count
32- machine.platform
Linux/x86_64- machine.python
3.14.7- machine.release
7.2.5-3-omarchy- reference.sha256
7b5e618e23b7f2b222665d83e99006eb0e9a51066e63e2e5ff8c51ae3c696c21- scorer.id
examples/summeval-judging/adapter.py- scorer.sha256
a886b5eac50f8ec28731d68262b01bd5d289715fa745f274d6823e1c3e932ac5