Chat History Bloats Context Windows; Microsoft.Extensions.AI Reducers Offer a Fix
Long support chats keep resending system prompts, tool-call noise, and old tangents on every request, which raises costs and can degrade model attention before hitting token limits. Naive approaches like keeping only the last N messages can break tool-using agents by separating function calls from their results, causing provider errors. Microsoft.Extensions.AI 10.10 introduces an experimental IChatReducer interface, a ReducingChatClient wrapper, and a UseChatReducer builder method to trim what the model sees while preserving the full transcript in a session store. The article describes count-based and token-aware reducers, plus a summarizing reducer that compresses older turns, and advises using a cheap model for summaries and keeping the system message intact. The full offline-tested project is promoted as a paid resource on a separate platform.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in