Testing pgvector, Postgres full-text search, and hybrid RRF on 30 RAG chunks

The author, while building a FastAPI and PostgreSQL RAG pipeline, faced problems with chunking that split financial tables and produced heavily overlapping chunks, plus slow and costly LLM-based evaluation. To address this, they built a local benchmark on PostgreSQL 17 and pgvector 0.8.7 using 30 enterprise chunks and 15 test queries, scoring retrieval with Recall@K, MRR, and NDCG computed locally at no cost. Results showed pure vector search struggled to rank exact error codes and SKUs at the top, pure full-text search failed on semantic paraphrases, and hybrid Reciprocal Rank Fusion performed best overall. The author also reported SQL fixes for stable hybrid search, including a deterministic tiebreaker and a relaxed iterative scan setting for filtered HNSW queries. The benchmark script, dataset, and chunk health checker were released as a free download.