RAG Pipelines: Embeddings Are Cheap, But These Stages Will Drain Your Budget

A developer building a Retrieval-Augmented Generation (RAG) pipeline set out to systematically measure costs at every stage after realizing common assumptions were misleading. Contrary to widespread belief among developers, embedding documents is one of the cheapest steps — a 1,000-page document costs roughly $0.18 to embed using OpenAI's current pricing. The real costs accumulate at the vector store infrastructure, which runs continuously at scale, and at the LLM inference stage, which consumes the most money and adds latency. A deduplication check — identifying unchanged documents before reprocessing — proved highly valuable, since roughly 80% of documents in a typical knowledge base remain the same between updates. Chunking and re-embedding only the sections that actually changed, rather than entire documents, was identified as the most effective way to reduce unnecessary pipeline costs.