More Context Can Make AI Answers Worse, Not Better, Research Shows

AI models advertise large context windows of up to one million tokens, but their effective working memory is significantly smaller in practice. As more text is added to a prompt, the model's attention — which is fixed and must be divided across all tokens — becomes diluted, reducing answer quality. Research, including the 'Lost in the Middle' study by Liu et al., found that information placed in the middle of long prompts is retrieved far less accurately than content near the beginning or end. Real-world data compounds the problem, as multiple similar or conflicting document versions make it harder for the model to identify the correct source. Experts recommend sending fewer but higher-quality chunks — typically three to five — and placing key instructions at the top and bottom of a prompt to improve reliability.