Developer finds three self-passing checks that were structurally incapable of ever failing
A developer at a fact-checking platform discovered three verifiers in their own codebase that always reported passing results regardless of actual correctness. One check compared a leap-year implementation against a line-for-line copy of itself, meaning both shared the same bug and agreed perfectly. A second verifier, on a page specifically about unfailing checks, constructed its expected result from the assertion count rather than the actual pass-or-fail verdict, displaying PASS 109/109 for two weeks while the real result was FAIL 106/109. A third check for Zipf's law used the wrong statistical estimator for two years, but all tests passed because the acceptable range was wide enough to accommodate any output the flawed estimator could produce. The author concludes that in all three cases, the assertion's expected value was derived from the very thing being tested, making failure structurally impossible.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in