How an ISP's WhatsApp billing bot fixed AI hallucinations without changing retrieval

A development team built a WhatsApp assistant for an internet service provider to handle plan and billing queries, and documented the real-world challenges they encountered. Unlike typical AI demos where wrong answers go undetected, billing responses are immediately verifiable by customers holding their invoices. The team found that poor retrieval was not the main problem; rather, the language model was narrating correct numbers inaccurately during response generation. To address this, they implemented a pipeline combining vector search, full-text search, and rank fusion, followed by a grounding verifier that escalates to a human agent whenever a response cannot be verified against source data. Their key insight was that structured output validation and grounding checks mattered far more than retrieval improvements alone.