Developer Builds AI Agent to Fix Weak Tests, Uncovers 8 Bugs in His Own Tool

A software developer built an AI agent over 30 hours to automatically generate tests that kill surviving mutation testing results, entering the micro1 Frontier Engineering Challenge among roughly 7,800 registrants. The project was motivated by a well-known gap in the industry: line coverage metrics show whether code ran, but not whether tests would catch errors if the code were wrong. While benchmarking 12 popular Python libraries, the developer found that 80 of 133 surviving mutants lived on code never executed by tests at all, not merely weakly tested. Widening test scopes up to 40 times barely changed this figure, suggesting the 'tests that run but don't assert' problem is largely specific to AI-generated tests rather than human-written ones. Before completing the agent itself, the developer discovered eight separate bugs in his own measurement harness, each capable of producing confident but incorrect results.