Vibes do not survive contact with production
Deterministic software has tests. AI features frequently ship with none, because their output is open-ended and writing assertions feels impossible. The result is a development loop where a prompt edit that improves one case silently breaks four others, and the regression arrives as a support ticket.
The fix is not perfect measurement. It is any consistent measurement, applied before the feature ships.
The three layers of an evaluation harness
Golden set. Twenty to two hundred real inputs with expected outcomes, drawn from actual usage rather than invented. Small and real beats large and synthetic.
Graders. Most output can be checked without a model:
- Schema validity — does the structured output parse and satisfy its contract?
- Constraint checks — required fields present, values within range, citations resolving to real sources
- Deterministic assertions — this classification must be one of five values
Only genuinely subjective properties (tone, completeness, faithfulness to a source) need a model as grader, and those should be scored against a written rubric.
Regression gate. The harness runs on every prompt, model, or retrieval change and reports a diff. Not a pass/fail — a diff. Which cases improved, which degraded, which are unchanged.
What to measure beyond accuracy
Accuracy is necessary and insufficient. Production systems live or die on:
- Refusal rate — how often the system declines when it should have answered
- Hallucination rate on grounded tasks — claims not supported by retrieved context
- Latency distribution — the p95, not the mean
- Cost per successful task — cost per call is vanity; cost per completed job is the number
- Escalation rate — how often a human had to intervene
Online evaluation closes the loop
Offline harnesses catch regressions. They cannot tell you what users actually needed. Production adds two more signals: explicit feedback and implicit behaviour — retries, abandonment, edits to generated output. Retries and edits are the most honest quality metric a system produces, and they cost nothing to collect.
Failures worth studying should be captured with full context — inputs, retrieved passages, resolved prompt, model version, output — and promoted into the golden set. That is how the harness compounds: every production failure becomes a permanent regression test.
Sequencing
Build the golden set and the graders before the feature is finished. It feels slower for a week and is faster within a month, because every subsequent change becomes measurable instead of debatable.