Production RAG pipeline costs are driven mainly by LLM generation, not vector storage

An illustrative model puts a production RAG pipeline at roughly $1,350 a month for 100,000 internal questions, $6,550 for 500,000 support queries, and $28,350 for 2 million enterprise searches, excluding engineering staff. The cost splits into index building and per-query work, and most spending falls on per-query work. Embedding a large corpus is comparatively cheap, while document parsing and OCR can cost more in compute and engineering time. Managed vector databases with usage-based pricing can see query costs rise with index size, and in the cited scenarios LLM generation accounts for 78–84% of monthly infrastructure spend. The author concludes that the most effective savings come from reducing the number of tokens sent to the model.