Why a passing idempotency test can mask a real concurrency bug in production

A developer discovered that a webhook handler for payment events sent duplicate notifications when two identical messages arrived within milliseconds of each other. The root cause was a classic read-modify-write race condition: both concurrent handlers read the same unchanged database row before either had written to it, causing both to trigger a payment notification. The original test appeared to cover the scenario but only ran a single handler against an already-updated row, meaning two overlapping calls never actually coexisted during the test. The fix required rewriting the test using Promise.all to simulate two handlers racing simultaneously, which correctly reproduced the production interleaving and exposed the bug. The case highlights how a confidently green test can be more misleading than a missing one if it tests the wrong execution path entirely.