Developer Tool 'canfail' Tests Whether CI Guards Can Actually Detect Failures

A developer has released canfail, a lightweight open-source tool designed to verify that CI pipeline checks are genuinely capable of catching errors, not just perpetually green by accident. The tool targets configuration files such as YAML, Terraform, and Dockerfiles — areas not covered by conventional mutation testing tools like Stryker. Users declare a specific code break, the expected failure output, and canfail applies the change, runs the check, and reports whether the failure was correctly detected. The tool enforces strict rules: the check must pass on a clean codebase first, failures must match the declared reason, and the modified file must be restored and verified by digest. A notable bug discovered during development revealed that stale Python bytecode cache could cause false positives, which was ultimately resolved using the PYTHONDONTWRITEBYTECODE environment variable.