How an AI customer service system quietly leaked personal data through overlooked paths

A developer building an agentic AI customer service system discovered that personal data could escape through three distinct boundaries, not one as originally assumed. While intake tokenization was applied to inbound customer emails, CRM tool results containing names, emails, and phone numbers were returned directly into the model's context window without any sanitization. Automated tests consistently passed because they only monitored the model's output — the least dangerous of the three data boundaries — leaving the return path from internal systems entirely uninstrumented. The system also grew unexpectedly from two runtime containers to twenty-seven services, with most of that complexity driven by unplanned requirements rather than new features. The author concludes that a copilot assisting a human agent can tolerate these gaps, but a fully autonomous customer service system cannot.