How One Team Built Automated LLM Evaluation to Catch 92% of Hallucinations Pre-Deployment

A development team shipping a RAG-based customer support assistant discovered critical flaws in their manual testing process only after over 500 users received hallucinated responses in production. The assistant had fabricated a billing policy and served API rate-limit figures sourced from a competitor's documentation. In response, the team built a production-grade LLM evaluation pipeline replacing ad-hoc human review with automated, domain-specific judges covering faithfulness, instruction-following, schema validation, and safety. The pipeline integrates directly into CI/CD workflows, blocking code merges that degrade response quality and enabling real-time regression detection. The new system, built around a versioned golden dataset and a multi-judge ensemble, now catches 92% of hallucinations before any model change reaches end users.