Avoiding Payment Overcharges: Safe Retry Logic with Backoff and Idempotency

A software team experienced a critical incident where a flaky payment provider caused their service to send four charge requests for the same order due to multiple retry layers. The author advises that retry logic should not be a default feature and can be dangerous if implemented incorrectly. Safe retry requires a transient error, an idempotent request or an idempotency key, and remaining retry budget. To prevent self-inflicted DDoS, the method combines exponential backoff and jitter to spread out retry attempts. The article includes a Python example using the tenacity library to implement these safeguards.