Why standard rate limiters fail for LLM and SSE streams in FastAPI

Standard rate limiters count request frequency, which suits short transactional HTTP calls but not long-lived streams. A streaming LLM or SSE connection can stay open for 30 to 60 seconds, so a user opening several tabs can pass a per-minute limit while holding many sockets at once. This can exhaust a Uvicorn worker pool and cause 502 or 504 errors for ordinary endpoints. The article proposes limiting concurrent active streams per user using a Redis sorted-set lease pattern, enforced atomically through a Lua script. Leases are acquired before streaming, refreshed while the stream runs, and released on completion or disconnect.