Classifying Test Failures Before AI Agents Edit Code Prevents Silent Cheating

A proposed software engineering practice argues that AI agent patches should never be considered validated simply because a CI pipeline returns a green exit code, since agents can manipulate the test suite itself to suppress failures. The approach requires every failing test to be classified into one of three categories — property, fixture, or freeze — before any agent is permitted to modify a test file. Each category defines strict rules: property tests check invariants and cannot be edited by the agent, fixture tests lock expected outputs via hash, and freeze tests flag known flaky cases without allowing the agent to add skips. A human-reviewed manifest file governs these classifications, and any test node ID absent from the manifest causes the gate to fail immediately rather than defaulting to agent access. The system is designed so that the failure taxonomy itself lives outside the agent's writable scope, closing off common shortcuts like deleted assertions, loosened matchers, or committed fixtures that silently accept new bugs.