Token Bucket vs Sliding Window: How to Build Rate Limiters That Hold Under Load

Rate limiting algorithms like token bucket and sliding window each have distinct failure modes that simple load tests often miss. The fixed window counter, a common first approach, can allow twice the intended request limit at window boundaries due to timing gaps. Token bucket implementations require monotonic clocks and thread-safe locking to prevent race conditions that silently over-permit requests. Unlike fixed or sliding window counters, token bucket natively supports variable request costs, making it more flexible for mixed endpoint traffic. In multi-instance deployments, per-process limiters fail to enforce global limits, requiring shared state via tools like Redis with atomic operations to remain effective.