Developer Fixes Local AI Degradation by Replacing Chat Logs with Distilled Facts

A developer running a personal AI assistant on a local Qwen3-4B-4bit model via Swama found that performance degraded significantly after 196 accumulated messages, with the model repeating itself and over-relying on stale context. The root cause was a memory layer that replayed raw conversation history into the prompt, a method that worked with hosted models but overwhelmed the smaller local one. The fix involved splitting memory into two distinct layers: short-term conversation state stored in RAM with an idle timeout and hard message cap, and long-term memory capped at 30 distilled personal facts per user. Fact extraction runs in a background thread using a hosted model to ensure quality without slowing down responses. The developer concluded that raw transcript replay is one of the easiest ways to degrade a personal assistant, and that minimal, structured memory outperforms large context dumps for small local models.