Engineer shares hard lessons from building an AI pipeline for 10,000 daily job listings

A software engineer built an LLM-powered scoring pipeline for a job platform that processed over 10,000 listings per day, but a parallel AI rewrite feature was shut down after API costs reached $3,000 per month for a single feature. The engineer found that using raw prompts in production caused unreliable outputs, including fabricated salary data, which was resolved by switching to OpenAI's function calling with strict JSON schemas. Cost management proved critical at scale, with GPT-4o mini and OpenAI's Batch API cutting expenses by 50% compared to synchronous calls. Further testing of DeepSeek V4 Flash showed comparable output quality at roughly 23 times lower cost, a gap the engineer described as the difference between a pipeline that ships and one that gets cancelled. The rewrite pipeline remains offline pending evaluation, highlighting how cost architecture — not just technical performance — determines whether AI features survive in production.