DSPy compiled programs silently resend up to 20 few-shot demos on every API call

DSPy, a framework that automates prompt optimization, attaches up to 20 few-shot demonstration examples per predictor module by default during its compilation step. These demos — including full reasoning traces — are converted into chat message pairs and prepended to every single inference call, not just the first. In a multi-module pipeline, this can mean 60 to 80 extra messages sent with each end-user request, adding significant and invisible token overhead. Developers are unaware of this cost because the demos are injected by the optimizer, not written in source code. The article advises explicitly setting lower demo budgets at compile time and auditing attached demos before deploying a compiled program to production.