Silent billing bug charged customers nothing for weeks despite green status
A software team discovered that their nightly billing job had been silently failing for weeks, successfully running and exiting without errors but never actually charging customers for overages. The job was rejected by the payment provider each time because it sent the previous day's usage events, which were too old to be accepted — a rule requiring timestamps no older than one hour. A second bug involved usage being reported against a subscription with no meter attached, returning a success response while nothing reached the invoice. A third issue involved faulty overage arithmetic that was never tested with actual computed values, only checked for the presence of certain lines in the source code. The team identified all three flaws by verifying state directly from the integrated system rather than trusting API responses, and by writing tests that confirmed failures before being relied upon.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in