Replace Vague Agent Patch Test Plans with CI-Enforced Machine-Readable Contracts
Test strategies written for agent-generated patches are often ignored, since agents don't read them and reviewers under time pressure approve green builds. The author proposes shipping a small JSON contract with each patch, enforced by a linter that rejects patches with inadequate contracts. The contract specifies property checks, seeded reproducibility, locked fixtures, frozen flaky tests, and an expiry date for freezes. Human ownership is required for invariants the linter cannot verify. The reference linter is a stdlib-only Python script of roughly 60 lines, with four rules and a blast-radius table determining how much checking a patch needs.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in