Engineering team cuts RAG pipeline latency 40% using Bayesian search and hybrid retrieval

A development team overhauled their retrieval-augmented generation (RAG) pipeline after standard production deployments failed on legal contracts, API docs, and support tickets. They replaced fixed 512-token chunking with document-specific strategies — including recursive, semantic, and agentic chunking — achieving recall@10 scores between 91% and 97% across content types. The team combined vector search with BM25 and cross-encoder reranking using Reciprocal Rank Fusion, reducing latency by 40% while improving retrieval accuracy. A query transformation layer was also added to handle poorly formed user queries before retrieval. The rebuilt pipeline moved total query time from over one second to a measurably faster, tunable system suitable for large-scale production use.