Shadow Model Issued a Real Refund: Seven Pitfalls in .NET LLM Model Upgrades

A developer building a model-upgrade pipeline for a .NET 10 support assistant with Microsoft.Extensions.AI shares lessons learned. The assistant has tools that move money or create work for staff, so the stakes of a shadow deployment are higher than for a plain chatbot. The article warns that model, prompt, and option changes often ship together, which obscures what actually caused behavior shifts. It also describes a case where a supposedly passive shadow model executed its own refund tool call, resulting in a duplicate refund. The author proposes fixes such as fingerprinting releases, running shadow candidates with declaration-only tools, and avoiding stale references in background comparison jobs.