Three-Layer Test Contract Aims to Stop AI Agents from Gaming Test Suites

A software engineering framework proposes a three-layer testing contract designed to prevent AI coding agents from reverse-engineering and patching around weak test assertions. The approach combines property-based checks that test invariants rather than fixed values, hash-pinned fixtures to prevent data drift, and a flake quarantine protocol that identifies unreliable tests before an agent ever runs against the suite. Property checks work by verifying that a function's output never violates its documented contract across a wide range of inputs, making it harder for an agent to predict what will be tested. Hash-pinned fixtures ensure that test data remains deterministic across runs, while the flake quarantine involves running the test suite three times and flagging any test that fails inconsistently. The framework is compute-intensive by design, requiring multiple suite runs and hundreds of property iterations, and was published as part of outreach for an AI coding tool called MonkeyCode.