0
ProgrammingDEV Community ·
Long Context Windows Don't Make RAG Obsolete, Argues Developer
A DEV Community author contends that larger model context windows change how much material gets retrieved but do not eliminate the need for retrieval. Anthropic's guidance suggests that knowledge bases under roughly 200,000 tokens can be placed directly into a prompt, especially with caching, though it notes larger bases need scalable solutions. The author cites Google's documentation on per-query token costs and latency, and research such as "Lost in the Middle" and Chroma's "Context Rot" report showing that models struggle to use information buried in long inputs. The piece also argues that retrieval enables document-level updates and per-user access controls that a static stuffed prompt cannot easily provide.
