Why Filtered Vector Search Breaks Benchmarks and Production Systems

Most vector database benchmarks report impressive speed and recall figures using unfiltered queries, but real-world production systems almost always combine vector search with metadata filters like tenant IDs or date ranges. Adding such filters to approximate nearest neighbor (ANN) searches disrupts the underlying graph index, which was built assuming all data points are accessible, causing latency to spike and recall to drop silently. There are three known approaches to handling filtered vector search, with the two most common methods failing in opposite ways depending on how selective the filter is. A third, newer technique can actually use filters to speed up the search rather than hinder it. The article argues this mechanics gap is one of the most underexplored problems in modern retrieval systems, often only surfacing when users report vague complaints that search feels broken.