Dev shares hard lessons from deploying LLM features in production

A developer writing for DEV Community recounts building an AI-powered support ticket classifier for a client, only to face a series of failures after moving from demo to production. Issues included malformed model outputs crashing the backend and the LLM inventing non-existent priority categories, highlighting how non-deterministic AI outputs break standard software assumptions. The author resolved output reliability by separating model inference from structured data extraction, letting deterministic code enforce schemas rather than relying on prompts alone. Context window overflows in production led to a shift toward vector database retrieval, cutting token costs by 60% and improving response quality. The key takeaway is that production AI integration demands the same engineering rigour as any data system — including validation layers, context management infrastructure, and quality monitoring beyond standard HTTP error tracking.