Contract AI Agent Failed Three Times, Exposing Gaps Between Validation and Real Accuracy

An enterprise legal team deployed a contract-extraction AI agent that initially showed 97% schema validation success, but broke three times in distinct ways after rollout. The first failure revealed that schema validation confirms output structure, not content correctness, after a table-formatted renewal clause caused a two-year date error that still passed validation. The second failure exposed a retry paradox, where the system filled missing fields with plausible but incorrect model-generated defaults, silently producing wrong outputs until flagged by the legal team weeks later. A third failure occurred when contracts from a newly acquired subsidiary — unseen during development — caused extraction accuracy to drop from 94% to 61%, illustrating the problem of distribution shift. The team concluded that being 'operator-ready' means an agent must handle unexpected real-world inputs reliably, not just perform well on a controlled test set.