Developer builds SSE coalescing middleware to fix Claude Code streaming delays
A developer running Claude Code through a self-hosted proxy called claude-code-router (CCR) encountered severe streaming delays, where approval prompts would stall for up to an hour due to a VS Code extension unable to process one-token-per-event streams fast enough. The root cause was traced to the third-party AI provider emitting highly granular Server-Sent Events, which overwhelmed the closed-source extension's processing pipeline. To fix this, the developer wrote a middleware layer that merges consecutive streaming delta events into larger chunks before they reach the extension, mimicking the format already used by Anthropic's official API. The middleware follows a strict rule: only merge events sharing the same block index and delta type, flushing the buffer immediately when any other protocol event appears, to avoid corrupting stream semantics. Configurable via environment variables like CCR_SSE_COALESCE_MS, the middleware also supports per-content-type timing overrides to balance latency for human-visible text versus background thinking deltas.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in