Why In-Memory Rate Limiting Breaks at Scale and How to Fix It

Rate limiting (throttling) controls how many requests a client can make within a given time window, protecting services from abuse and overload. A common production failure occurs when scaling horizontally: each pod maintains its own in-memory counter, so a 100 req/min limit effectively becomes 800 req/min across 8 pods with no errors logged. The solution requires moving counters to a distributed store like Redis, which then raises the question of which algorithm to use. Popular options include fixed window counter, sliding window log, sliding window counter, and token bucket, each with different trade-offs around burst handling, memory usage, and boundary-edge accuracy. For distributed token bucket implementations on Redis, atomic Lua scripts are essential to avoid race conditions in read-modify-write operations.