Product Quantization: How AI Systems Compress Millions of Vectors to Save Memory

Storing large-scale vector embeddings used in AI applications demands significant memory — for example, 10 million 768-dimensional float vectors require roughly 30 GB of storage. Product Quantization (PQ) is a compression technique, supported by the FAISS library, that reduces this footprint without discarding all the original information. PQ works by splitting each high-dimensional vector into smaller sub-vectors, then replacing each sub-vector with the ID of its closest match in a learned reference set called a codebook. This means a full floating-point vector can be approximated by a compact list of small integer IDs, drastically cutting storage requirements at scale. The approach enables faster and cheaper similarity searches across millions of embeddings, making it especially useful for retrieval-augmented generation (RAG) and other large-scale AI applications.