How One Team Built Automated LLM Evaluation That Catches 92% of Hallucinations

A development team discovered the cost of informal AI testing after their RAG-based customer support assistant served hallucinated responses to over 500 users in production. The assistant had fabricated a billing policy and cited a competitor's API rate limits, exposing a complete absence of automated evaluation. In response, the team built a production-grade LLM evaluation pipeline integrating multiple judge types — including faithfulness, instruction-following, JSON schema, and safety checks — into their CI/CD workflow. The system uses a versioned golden dataset and a judge ensemble to automatically score responses before deployment, blocking merges that degrade quality. The new pipeline now catches 92% of hallucinations before any code reaches production.