The rule. Treat evaluations the way design teams treat usability tests: encode what good AI behavior looks like as graded test cases, run them before shipping, and rerun them at every prompt or model change.
Why. AI features change under their users: models learn, adapt, and get swapped out, which is why the 18 guidelines for human-AI interaction give "over time" its own phase of seven guidelines, including "update and adapt cautiously" and "notify users about changes" (Amershi et al., 2019). Without evals, teams learn about behavior drift when users report the agent feels worse, then debug reactively with no way to verify fixes except guess and check (Anthropic, 2026). The practice is not engineering-only: Anthropic's guidance states that the people closest to product requirements and users are best positioned to define success, and that product managers can contribute eval tasks directly (Anthropic, 2026).
Seen in the wild. Descript built evals for its video-editing agent around three product dimensions (don't break things, do what I asked, do it well), with grading criteria defined by the product team and periodic human calibration (Anthropic, 2026).
Verified as of 2026-08-27 against current Anthropic documentation and engineering guidance.