Context Optimization Is Not Stream Recovery: Integrating context-mode Without Hiding Failures

An engineer describes a production agent that failed because verbose tool output, such as a long pytest log, filled the model's context window and left it reasoning from truncated evidence. The project context-mode addresses this by sandboxing tool output and persisting session memory outside the immediate inference context. The author stresses that context reduction and stream delivery must be kept architecturally separate, since a generic truncator can strip critical error lines or corrupt tool-call relationships. Instead, the recommended approach enforces an application-level contract so that only bounded, relevant tool results reach inference, while complete output stays retrievable. The article also presents a Next.js App Router handler using the Vercel AI SDK 5 family and an OpenAI-compatible gateway, noting that unbuffered streaming must be measured rather than assumed.