Developer finds 61 of 400 automated tests had zero assertions, all marked passing
A software developer discovered that 61 out of 400 automated tests in their suite contained no assertions whatsoever, yet all were recorded as passing in CI pipelines. Beyond the empty tests, the developer identified four common patterns of deceptive assertions, including soft asserts missing the required assertAll() call and tautological checks that could never fail. These hollow tests were routinely reported to management as evidence of quality coverage, with test count — not test effectiveness — being the key sprint metric. The root cause, the developer argues, was that ticket definitions of done required only that a test exist and pass, never that it actually prove correct software behaviour. The incident highlights how optimising for test quantity over test validity can create a false sense of product reliability.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in