How One Team Built Automated LLM Evals That Catch 92% of Hallucinations Pre-Deploy

A development team discovered critical gaps in their AI quality process after a RAG-based customer support assistant served hallucinated responses—including fabricated policies and competitor data—to over 500 users in production. The failures stemmed from an entirely manual review process where engineers approved outputs by simply reading a handful of answers before release. In response, the team built a production-grade evaluation pipeline integrating automated judges for faithfulness, instruction-following, JSON schema validation, and safety checks directly into their CI/CD workflow. The system uses a versioned 'golden dataset' of test cases run through a judge ensemble, enabling regression detection and blocking code merges that degrade response quality. The team reports the pipeline now catches 92% of hallucinations before deployment, replacing subjective human review with measurable, domain-specific metrics.