Why pgvector, 768 Dims, and Gemini Flash: RAG Design Decisions Unpacked

A technical breakdown of a Retrieval-Augmented Generation (RAG) pipeline explains the reasoning behind key architectural choices, including using pgvector over dedicated vector databases like Pinecone or Weaviate. The author chose pgvector because it integrates with existing PostgreSQL infrastructure, supports SQL and vector search in a single query, and handles millions of documents via HNSW indexing. Google's gemini-embedding-001 model was configured to output 768 dimensions instead of the default 3072, balancing retrieval quality with pgvector's 2000-dimension HNSW limit and storage efficiency. Separate task types — RETRIEVAL_DOCUMENT for ingestion and RETRIEVAL_QUERY for querying — were used to leverage the model's asymmetric training, which improves retrieval accuracy. The HNSW index was preferred over IVFFlat for its faster query speed and higher accuracy at scale, while Gemini 2.5 Flash was selected as the answer-generation model.