Why Running Code Matters More Than Reviewing It: A Testing Lesson

A developer discovered a critical gap between code review verdicts and actual test evidence while working on a notification queue ordering change. Although a review had returned a 'pass-with-risk' result with a named reason, the requirement had never actually been executed, leaving the risk labelled but unverified. When the code was finally run, a real timing failure emerged that review alone could not have detected — the scheduler incorrectly prioritised one notification over another under a specific condition. The experience highlighted that a skipped test row looks identical to a passing one in a results matrix, making unverified requirements invisible unless checks are explicitly enforced. The developer concluded that test matrices should preserve records of both failures and fixes, since deleting resolved failures erases the only proof that a check was ever run.