RAG Architecture Explained: Five Boxes, Six Arrows, and Hidden Costs

A Retrieval-Augmented Generation (RAG) system can be broken down into five core components: an index, a retriever, a reranker, a context builder, and a loop controller. Each connection between these components carries a data payload that incurs costs both when data is moved and when it is processed. The index sets a hard ceiling on system performance, since any fact lost during chunking or indexing cannot be recovered downstream by reranking or prompt engineering. The context builder is the only component that can actively reduce costs within a single request, while the loop controller poses the greatest billing risk because each additional iteration re-runs all upstream components. Engineers are advised to map every data transfer across their stack and price the largest one — the input into the model prompt — to estimate baseline monthly retrieval costs before any optimization.