How to Build a FastAPI Endpoint That Streams LLM Responses to the Browser

A technical guide demonstrates how to build a streaming chat endpoint using FastAPI and httpx that delivers large language model responses token-by-token to the browser. The setup relies on an async generator as the response body and a shared httpx client initialized at app startup to preserve connection pooling and avoid repeated TLS handshakes. Server-Sent Events are structured with three named event types — token, error, and done — to help clients cleanly distinguish between streamed content, failures, and completion signals. Key headers such as X-Accel-Buffering and Cache-Control are set to prevent intermediate proxies or servers from buffering the stream. Since the browser's native EventSource API only supports GET requests, real-world chat UIs are advised to use the Fetch API with a stream reader for POST-based endpoints.