Mutation Testing Reveals Weak Characterization Tests Through a Kill-Rate Gate

A passing characterization suite shows only that tests ran against current code, not that they would catch behavioral changes. The author proposes measuring a suite's kill rate by deliberately breaking functions and checking whether the tests fail. Because such tests often assert weak properties like non-None results, almost any edit can slip through unnoticed. Before measuring, the suite should be run twice on untouched code, since flaky baselines would corrupt every mutation result. The post then presents a Python 3.9+ script that applies three cheap, mostly semantics-changing mutations (flipping comparisons, swapping and/or, bumping integer constants) to one function at a time in a temporary copy of the repository. The script restores original bytes after each run because the AST round-trip discards comments and formatting.