---
name: your-org/your-benchmark
version: 0.1.0
description: One line. What this spec holds an agent to.
maintainers: [your-org]
license: CC-BY-4.0
forked_from: null
tasks:
  - {id: first-group, name: First task group}
  - {id: second-group, name: Second task group}
---

## Evals

### an-eval-id
task: first-group
severity: blocking

**One expert-written sentence a practitioner would recognize.**

reads: the evidence the judge may see — name the documents and whether it
  reads the trajectory and final answer
fail-when: the concrete condition under which this eval fails. Write it so
  two reviewers reading the same output reach the same verdict. Carve out
  what belongs to other evals.
na-when: the condition under which this eval does not apply. "Never" is a
  valid answer.
derived_from: [where-this-rule-comes-from]

### a-second-eval-id
task: second-group
severity: major

**A second sentence — one rule per eval, sentence and check travel together.**

reads: …
fail-when: …
na-when: …
derived_from: [drafted-your-observation]

## About

What this spec is for, where the evals come from (cite your sources),
what the corpus is and how it was produced (declare synthesized data),
and how verdicts are produced. See the format definition at
https://recordroom.ai/specs/FORMAT.md — corpus layout:

    your-benchmark/
    ├── SPEC.md
    ├── corpus/
    │   ├── tasks/<task-id>/   task.json + material files
    │   ├── outputs/<id>.json  trajectories: turns + final
    │   └── expected.json      draft answer key (never judge evidence)
    └── dist/                  built artifacts + runs

License your spec CC-BY-4.0 so others can fork it, argue with a sentence,
and adapt the thresholds to their shop — that argument is the product
working.