model
eval-llama-maverick llm
A model record: onboarded once for the project and used by any eval's arms.
- weights
- open
- licence
- Llama, gated
- aliases
eval-llama-maverick- settings
- defaults
- onboarding
- not onboarded:
evh model-onboard - best result per goal
- #3 of 29 Short news summaries (CNN)
#7 of 19 Evidence retrieval for scientific claims (SciFact)
#12 of 28 Named-entity extraction (Few-NERD)
#15 of 28 Encyclopedia topic classification (DBpedia)
#18 of 33 A scorer that agrees with experts on summaries (SummEval)
#26 of 30 News topic classification (AG News)
01deployments and their onboarding
No deployment is recorded for this model. Not onboarded yet: evh model-onboard eval-llama-maverick.
02settings history
No change of settings or build is recorded.
03arms
| arm | deployment | goal | its settings | arm smoke | result |
|---|---|---|---|---|---|
| ag_llama_l_n200_v1 · adjudicated ag_news_200 | — | — | max_tokens 200, temperature 0.0 | none | correct 0.925 |
| cnn_llama_l_n200_v1 · judged/judge-summ-pick-v1 cnn_dailymail_200 | — | — | max_tokens 1200, temperature 0.0 | none | judge_won 0.003 |
| cnn_llama_l_n200_v1 · judged/judge-summ-v1 cnn_dailymail_200 | — | — | max_tokens 1200, temperature 0.0 | none | judge_score 4.700 |
| cnn_llama_l_n200_v1 · judged/judge-summ-v2 cnn_dailymail_200 | — | — | max_tokens 1200, temperature 0.0 | none | judge_score 4.529 |
| sf_llama_l_n200_v1 · repeats scifact_200 | — | Evidence retrieval for scientific claims (SciFact) | max_tokens 700, temperature 0.0 | none | ndcg_10 0.715 |
| cnn_llama_l_n200_v1 · silver cnn_dailymail_200 | — | — | max_tokens 1200, temperature 0.0 | none | coverage 0.370 |
| ag_llama_l_n200_v1 ag_news_200 | — | News topic classification (AG News) | max_tokens 200, temperature 0.0 | none | correct 0.840 |
| cnn_llama_l_n200_v1 cnn_dailymail_200 | — | Short news summaries (CNN) | max_tokens 1200, temperature 0.0 | none | coverage 0.339 |
| db_llama_l_n200_v1 dbpedia_280 | — | Encyclopedia topic classification (DBpedia) | max_tokens 200, temperature 0.0 | none | correct 0.975 |
| fn_llama_l_n200_v1 few_nerd_280 | — | Named-entity extraction (Few-NERD) | max_tokens 600, temperature 0.0 | none | f1 0.599 |
| sf_llama_l_n200_v1 scifact_200 | — | Evidence retrieval for scientific claims (SciFact) | max_tokens 700, temperature 0.0 | none | ndcg_10 0.728 |
| se_llama_l_n100_v1 summeval_100 | — | A scorer that agrees with experts on summaries (SummEval) | max_tokens 1500, temperature 0.0 | none | tau 0.454 |