How One Developer Stabilized a RAG Pipeline Processing 10,000+ Job Listings Daily

A developer building a job board RAG pipeline discovered that strategies working in staging often fail under real production load when processing thousands of listings daily. Chunking strategy proved critical from day one, affecting embedding quality, retrieval accuracy, and cost — with recursive character splitting at ~400 tokens with overlap ultimately outperforming fixed-size and semantic approaches. Pre-processing raw ATS output from platforms like Greenhouse and Lever into a consistent format before chunking significantly improved results. On the embeddings side, OpenAI's text-embedding-3-small outperformed a self-hosted Llama 3.1 model, which struggled with domain-specific job market terminology and produced noisier retrieval. The core takeaway is that production RAG requires tailoring chunking and embedding choices to the specific document type rather than relying on generic defaults from tutorials.