Developer Cuts LLM Invoice Extraction Errors Using Schema Validation and Retry Logic

A software developer building a PDF invoice extraction system found that GPT-4 hallucinated data roughly 30–40% of the time when given a simple prompt, producing wrong field names, malformed dates, and even fabricated line items. Prompt engineering improvements and few-shot examples raised accuracy to around 80%, but failures persisted with unusual document layouts. The developer identified the root cause as treating the LLM as a black box rather than separating extraction, validation, and correction into distinct steps. By defining a strict Pydantic data model and using OpenAI's structured output mode, the system could immediately validate each response against a schema. When validation failed, the error message was fed back to the model as context for an automatic retry, significantly reducing hallucinations in production.