Guide to two-stage retrieval and reranking for balancing recall, precision, and latency

The guide explains how to separate candidate retrieval from relevance reranking in search-and-generation systems. It describes a funnel in which a fast first stage pulls a broad set of candidate passages and a slower reranker narrows them to the most relevant few. A table maps common symptoms, such as low answer relevance, high latency, token budget overruns, and missed clauses, to likely root causes and checks. The article also outlines component responsibilities, latency targets, and deterministic filtering rules, along with a sample function for tuning top-N and top-k within token and latency limits. It closes with monitoring practices such as tracking hallucination rates and keeping configuration documentation in sync.