How One Developer Cut AI Costs 50% While Scaling a Job Board to 10,000 Daily Listings

A software developer building a job board platform encountered severe cost overruns after scaling an LLM-based listing scorer from 1,000 to 10,000 daily entries using real-time GPT-4 API calls. To address this, the developer switched to OpenAI's Batch API, which processes requests at half the cost of real-time endpoints and handles internal retries automatically. Structured JSON output was enforced using OpenAI's tool-calling feature, eliminating unreliable text parsing and ensuring consistent scoring fields including relevance score, category, and confidence level. The redesigned pipeline collects listings throughout the day, submits a single batch at midnight, and retrieves scored results by morning, cutting per-listing cost from $0.003 to $0.0015. The changes reduced monthly API spending by nearly $500 while improving reliability and removing the need for manual rate-limit and retry handling.