arm

sf_e5_base_n200_v1 baseline

One config on scifact_200, scored one way; its repeats averaged per item.

dataset: scifact_200
scored against: gold/scifact_200 gold
goal: Evidence retrieval for scientific claims (SciFact)
runs: 1
headline: ndcg_10 0.719
last run: 2026-09-28 17:46

Inspect it With the 26 arms scored the same way

01metrics

metric	mean	items	better
ndcg_10	0.719	200	higher
cost_usd	$0.000000	200	lower
duplicate_ids	0.000	200	neutral
found_any	0.955	200	neutral
latency_ms	59.4	200	lower
mrr_10	0.689	200	higher
n_relevant	1.135	200	neutral
n_returned	100.0	200	neutral
parsed	1.000	200	higher
rank_of_first	6.455	200	neutral
recall_10	0.831	200	higher
recall_100	0.954	200	higher
unknown_ids	0.000	200	neutral

02cost and speed

cost per item: $0.000000
speed per item: p50 56 · p95 75 ms

Per item, over its items: cost as dollars and tokens (a local model is free but still uses tokens), speed as the median and the slow tail (p95). A goal that names this arm weighs them in its decision.

03runs

run	made	items	cost	how
`sf_e5_base_n200_v1_20260928T174656Z`	2026-09-28 17:46	200	$0.000000	reparse

04config

config sf_e5_base_n200_v1 · adapter examples/retrieval-scifact/adapter.py · credentials are never shown

corpus_id: scifact
device: cpu
doc_prefix: passage:
encode_batch: 32
family: e5
model: intfloat/e5-base-v2
precision: fp32
price_tier: free
provider: dense
query_prefix: query:
top_k: 100

05deploy sheet

measured by: sf_e5_base_n200_v1_20260928T174656Z on scifact_200
against: gold/scifact_200
model sent: intfloat/e5-base-v2
model record: intfloat-e5-base-v2
server: not recorded
build: f52bf8ec8c7124536f0efb74aca902b2995e5bcd
settings sent: corpus_id: scifact
device: cpu
doc_prefix: passage:
encode_batch: 32
model: intfloat/e5-base-v2
precision: fp32
provider: dense
query_prefix: query:
top_k: 100
not sent: bookkeeping: family, price_tier
declared: deployment.corpus_id: scifact
deployment.corpus_n_docs: 5183
deployment.corpus_sha256: 297643e2d005109d9fc25d460c080faa45b0cc3ad881ea002941c67b6e41984e
deployment.doc_prefix: passage:
deployment.index_built_this_run: False
deployment.provider: dense
deployment.query_prefix: query:
deployment.scorer_sha256: 51d9e8bd57bc2e75ceccce80b7d60157c42f9959cadf3d7e9cf738f73fd10d04
deployment.tokenizer_sha256: 67a185c8c30efefa133188f15ba429328086f837a642a17e347a5a2ee437b79d
deployment.top_k: 100
deployment.weight_files: ['model.safetensors:437955512']
adapter: examples/retrieval-scifact/adapter.py 46cd2bd89061
harness: 0e20aaeb3dca

No capture here says what production ran (the adapter's production_ran).

Command: evh deploy-sheet --config sf_e5_base_n200_v1 [--capture C]

06conditions

data.items_sha256: b97dfe735a78eb4978cde41e2ca9a0bff424de9e6a0a2bdada5d167e3423004d
deployment.build: f52bf8ec8c7124536f0efb74aca902b2995e5bcd
deployment.device: cpu
deployment.model: intfloat/e5-base-v2
deployment.quantization: fp32
deployment.source: adapter
fingerprint.version: 5
library.numpy: 1.26.4
library.openai: 3.19.2
library.scikit-learn: 1.9.1
library.sentence-transformers: 3.4.1
library.torch: 2.2.2
library.transformers: 4.55.4
machine.cpu_count: 12
machine.label: mac-dev
machine.platform: Darwin/x86_64
machine.processor: i386
machine.python: 3.11.8
machine.release: 24.6.0
reference.sha256: b522e08017ebc7dd71c5627bccfdd688c5124096df8a5e6454684141bc361d99
scorer.id: examples/retrieval-scifact/adapter.py
scorer.sha256: 46cd2bd89061c9f3bff0493cce7ab292dd6f9b1bae7c7e482af0aebdd1479cee

07rounds

Round r1 of Evidence retrieval for scientific claims (SciFact), closed: one of its arms.