Writing Requirements for an Agent You Can't Fully Predict

Imagine a requirement that reads: “The support agent should resolve customer issues accurately and quickly.” Everyone nods. Then the build begins, and each person reads it differently. Engineering optimizes for speed. QA hunts for wrong answers. The product manager worries about tone. At launch, no one can say whether the requirement was met, because it never said how to tell.

With ordinary software, a vague requirement is a nuisance. With an agent, it is a blocker. The agent’s behavior varies from run to run, so the requirement is the only fixed point the team has.

Why agents change how you write requirements

Traditional acceptance criteria assume a deterministic system: given input X, output Y. That carries over for parts of an agent, such as “never issue a refund over a set amount without approval.” Those are contracts, and software engineering handles them well.

What does not carry over is the rest. For an agent, “given input X” produces a range of outputs, some better than others. From machine learning you borrow the idea that quality is a rate measured on a dataset. What neither field hands you is a way to write the decision down in plain language that a product manager, a tester and an executive can all sign.

The practice: four kinds of statements

Sort every requirement into one of four buckets.

1. Hard constraints. Things that must hold on every run. “Never reveal another customer’s data.” “Never take a destructive action without confirmation.” These are checked pass/fail, and a single violation matters. Prefer to enforce them in code, outside the model.
2. Quality targets. Things measured as a rate. “At least 90% of refund questions answered correctly, as judged by rubric v2, on the refund evaluation set v3.” Note the three parts: the threshold, how it is measured, and on what data.
3. Trade-offs. Cost and latency limits that pull against quality. “Median response under 6 seconds and average cost under 4 cents per conversation, provided quality targets are met.” State which wins when they conflict.
4. Unknowns. What happens when you lack evidence. Decide in advance what “insufficient evidence” means (for example, fewer than 100 cases in a category, or too much run-to-run variation to tell) and what the team does about it: collect more data, narrow the launch, or hold.

Then add three things for each requirement:

• An owner who accepts or rejects the evidence.
• A review point: when the threshold is revisited. Real traffic changes.
• A source of truth for the data: which dataset version defines “the refund questions.”

A note on thresholds: choose them from the cost of being wrong, not from a round number. If a wrong refund answer costs a support escalation, 90% might do. If it costs money or trust, the bar is higher, and the hard constraints do more of the work.

A worked example (illustrative)

Here is a requirement sheet for an imagined support agent. All thresholds are invented for illustration.

ID	Type	Statement	Measured by	Owner
R1	Hard	Never disclose another customer’s data	Automated check on every test run; any failure blocks release	Architect
R2	Hard	Refunds above the approval limit are routed to a human	Tool-call check, every run	QA
R3	Quality	At least 90% of refund questions correct	Rubric v2, refund set v3 (200 cases), 5 runs, lower bound of the range	Product manager
R4	Quality	At least 85% of order-status questions correct	Same method, status set v1	Product manager
R5	Trade-off	Median latency under 6 s; cost per conversation under a stated cap	Measured in the same runs as R3	Leader
R6	Unknown	Category with fewer than 50 cases is reported “insufficient evidence”, not “passed”	Dataset count report	QA

Two details deserve attention. R3 asks for the lower end of the range across runs, not the best run, because a launch decision should be based on a pessimistic reading. And R6 gives the team a legitimate third answer besides pass and fail.

Pseudo-code, equally illustrative, for how a release check could read this sheet:

for each hard constraint: any failure in any run -> BLOCK
for each quality target: lowest run result >= threshold -> PASS
                         enough cases?  no -> INSUFFICIENT EVIDENCE
for each trade-off: apply only if quality targets pass
decision owner signs the result, with the open items listed

Making the sheet work on Monday

1. Take your current “should be accurate and fast” sentence.
2. Split it into the four buckets above. Expect to find that hard constraints are fewer than you thought and unknowns are more.
3. For every quality target, fill in threshold, measurement method and dataset. If the dataset does not exist yet, writing that down is the first task.
4. Ask QA to try to break each line: could this requirement be met while the user is still unhappy? Fix the gaps.
5. Name the decision owner for each line.

What this doesn’t solve

• Thresholds are judgment calls. The sheet makes them explicit and reviewable. It cannot make them correct.
• Evaluators are imperfect. If rubric v2 is wrong, R3 passes on a mistake. The sheet should name how the evaluator is checked against human review.
• Datasets drift. A set that represented users in October may not in March. The review point exists for this reason.
• Rates need enough cases. A handful of examples support only rough statements. Say so.
• Methods are young. Writing requirements this way is a reasonable practice, not a proven one. Evidence that would support it: teams using such sheets showing fewer disputes at release and faster detection of regressions than comparable teams who do not.

---

See it in practice

Traigent publishes a Python SDK for running experiments against requirements like R3 and R5. Per its public README, you declare objectives such as accuracy, cost and latency, point it at an evaluation dataset, and it runs trials across configurations. It supports balancing several objectives at once and custom success metrics. See traigent.ai.

---

Who this is for: product managers, engineering leaders and QA engineers who define and sign off what an agent must do.