Engineering team cuts RAG pipeline latency 40% using Bayesian search and hybrid retrieval

A development team overhauled their Retrieval-Augmented Generation (RAG) pipeline after standard production deployments proved insufficient for legal, API, and support-ticket use cases. The team replaced fixed 512-token chunking with document-type-specific strategies, including recursive, semantic, and agentic chunking, achieving recall@10 scores between 91% and 97% across content types. They combined vector search with BM25 and a cross-encoder reranker using Reciprocal Rank Fusion, which improved relevance correlation from roughly 0.75 to 0.92. Query transformation was also introduced to handle poorly formed user queries before retrieval. Together, these changes reduced end-to-end query latency by 40% while pushing overall recall@10 to 95%.