cnn_bart_m_n200_v1 · judged/judge-summ-v2

01metrics

02cost and speed

cost per item

$0.000000 · 766 tok

speed per item

p50 4,649 · p95 7,223 ms

Per item, over its items: cost as dollars and tokens (a local model is free but still uses tokens), speed as the median and the slow tail (p95). A goal that names this arm weighs them in its decision.

03runs

04config

config cnn_bart_m_n200_v1 · adapter examples/summarization-cnn-dailymail/adapter.py · credentials are never shown

device

cpu

early_stopping

true

family

bart

length_penalty

2.0

max_length

142

min_length

56

model

sshleifer/distilbart-cnn-12-6

no_repeat_ngram_size

3

num_beams

4

precision

fp32

price_tier

free

provider

hf_local

05conditions

data.items_sha256

257feda902391b934b8cde940c15e91201096dbe96ec5a81337e9da007ce3ec7

deployment.device

cpu

deployment.model

sshleifer/distilbart-cnn-12-6

deployment.quantization

fp32

deployment.source

adapter

fingerprint.version

5

library.numpy

2.4.6

library.openai

3.19.2

library.rouge-score

0.1.2

library.torch

2.14.0

library.transformers

4.55.4

machine.cpu_count

4

machine.platform

Linux/x86_64

machine.processor

x86_64

machine.python

3.11.15

machine.release

6.18.44-fc-v50

reference.sha256

5d0b02c3798017535572d24dd3ce0592332938a0c91e93684d163963a60a7cd9

scorer.id

judge-score:judge-summ-v2

scorer.sha256

4b5440bf0599fd12fd2d69a2697f1b0cdf54871486ef228b60be554750e82887

06the same config, scored other ways

scored against silverscored by a judgewrote the reference it is scored againstscored against a label set, not a reference setunmarked rows are scored against gold