How Staff Engineers Solve Cache Stampede: The 40,000-Request Database Problem

A cache stampede, or 'thundering herd,' occurs when a popular cache key expires and thousands of simultaneous requests bypass the cache and hammer the database with identical queries. The core issue is duplication, not volume — during a 200ms cache rebuild window, thousands of requests each independently trigger the same database query. Engineers can address this using either a distributed lock, where only the first request rebuilds the cache while others wait, or a stale-while-revalidate strategy, which serves slightly outdated data instantly while refreshing in the background. The right choice depends on the product context — account balances demand fresh data via locking, while product listings can tolerate brief staleness. A further safeguard is adding random jitter to cache TTLs at write time, preventing mass simultaneous expiries when many keys are set together.