Staged Retrieval and Deterministic Keys Prevent Duplicate Candidates in Search

The article argues that deduplication should be built into retrieval design for recruiting candidate search rather than handled as a cleanup step after a model responds. It recommends staged retrieval with explicit collections, bounded queries, and a stable source key derived from tenant, document ID, and content version. A retrieval result should carry identifiers, access-control metadata, source references, chunk text, and a score so reviewers can trace ranking changes. The author compares Elasticsearch, Pinecone, Weaviate, and a thin vector API called Infrai, noting each option's operational trade-offs. Because queues are at-least-once, the piece stresses consumer idempotency so that retries after worker timeouts update existing chunks instead of creating duplicate candidate hits.