Product requirements (PRDs)¶
Purpose¶
A PRD says what the harness does for the person using it, and why: the problem, who has it, what they need to be able to do, and what is deliberately left out. It captures:
- User needs and the situations they come from;
- Functional requirements, numbered (FR1, FR1.1, ...), each one checkable;
- Non-goals, so a later reader does not "fix" a deliberate omission;
- Links to the experience (UXS), the technical design (RFC) and the decisions (ADR).
Status vocabulary: a PRD is Draft while it is being written, Partial (vX.Y.Z) when
some of it has shipped, and Implemented (vX.Y.Z) when all of it has. Completed and
Accepted are for RFCs and ADRs, not PRDs. Before a release, a PRD whose last part lands in
it says Implemented (X.Y.Z, unreleased) until the tag.
How the documents fit together¶
New work starts with a PRD. From 0.4.0 on, the order is:
- PRD: what the user needs and why (this index).
- UXS: what the user sees and types, page by page and command by command, when the work has a surface (UX specifications).
- RFC: how it is built: data, interfaces, algorithms, alternatives (proposals and designs).
- ADR: a few lines recording a key decision and pointing at the documents above (decisions).
The PRDs below were written for 0.4.0 from the code as it is, so each describes shipped behaviour. The release notes (CHANGELOG) say what changed when.
PRDs¶
| PRD | Title | Status | Related | What it covers |
|---|---|---|---|---|
| PRD-001 | Running evals and comparing arms | Implemented (v0.4.0) | RFC-001, RFC-002, UXS-003 | Frozen datasets, arms, runs with a fingerprint, the arm's own noise, paired tests with Holm, baselines with a reason |
| PRD-002 | Labels, and the reference sets promoted from them | Implemented (v0.4.0); labels and promotion built for 0.6.0 | RFC-023, ADR-0022, RFC-002 | Labels with their source (person, model, rule); a reference is labels promoted on purpose at the tier you choose; scoring against labels without promoting; re-scoring for $0 when the reference changes |
| PRD-003 | Re-measuring with a validated LLM judge | Implemented (0.3.0) | RFC-005, ADR-0001 | Judge scorers, the validation gate, two vendors, blind and stored verdicts |
| PRD-004 | The catalog and the run inspector | Implemented (v0.4.0; FR6, the views for an agent: v0.5.0); the Captures page (0.6.0); labels, picks, the sheet planned for 0.6.0 | UXS-001, UXS-002, RFC-003, ADR-0002 | Pages over the runs, an inspector that tests arms, read-only, every view also a command |
| PRD-005 | Goals, and eval rounds | Implemented (v0.4.0); the deploy sheet built for 0.6.0 | RFC-004, UXS-001, ADR-0004 | What an evaluation is for and the measure that decides; optional rounds with a hypothesis, retirements and a conclusion |
| PRD-006 | Datasets, versions and item fields | Implemented (v0.4.0) | RFC-006, ADR-0003 | Versions with lineage, typed item fields outside the identity, items at a pinned URL |
| PRD-007 | One install, your own keys | Implemented (v0.4.0); several projects in parallel (0.6.0) | UXS-003 | The evh package, the project's own data, the adapter seam, hosted providers with keys you bring |
| PRD-014 | Datasets from production, and evaluating what production did | Built (0.6.0) | RFC-019, RFC-023, ADR-0019, ADR-0023 | Importers, captures sampled as they are read, datasets from a capture, production as an arm, labels from outcomes, the sheet, picks, the next version of a governed dataset, text kept off public surfaces |
| PRD-015 | Models, onboarded once, for every eval | Built (0.6.0; written after the build) | RFC-020, RFC-021, ADR-0021 | Model records and deployments, onboarding, arm smoke, the Models page, fingerprint v5, and their limits as built |
Adding one¶
Name it PRD-NNN-short-slug.md (next number, three digits) with the header used here
(Status, Authors, Related), then the sections Summary, Background & Context, Goals,
Non-Goals, Personas, User Stories, Functional Requirements, Success Criteria and Related
Documents. Add its row to the table.