How One Team Built Automated LLM Evaluation That Catches 92% of Hallucinations

A development team discovered the cost of relying on informal, manual testing after their RAG-based customer support chatbot served hallucinated responses to over 500 users in production. The assistant fabricated a billing policy and cited a competitor's API rate limits as its own, exposing a complete absence of automated quality checks. In response, the team built a production-grade evaluation pipeline that runs domain-specific judges — covering faithfulness, instruction-following, JSON schema validation, and safety — within CI/CD workflows. The system uses a versioned golden dataset and a judge ensemble to catch regressions before any code is merged, blocking deployments that degrade response quality. The approach, detailed in a technical write-up on DEV Community, now catches 92% of hallucinations before they reach end users.