Study finds benchmark answers may be leaking into large language model training

A new analysis suggests that large language models (LLMs) may already contain benchmark answers within their training data, raising concerns about the validity of AI performance evaluations. The phenomenon, known as data contamination, means models could be recalling memorized answers rather than demonstrating genuine reasoning ability. This calls into question the reliability of widely used benchmarks that the AI industry depends on to measure model progress. The findings highlight a growing challenge for researchers trying to accurately assess how capable AI systems truly are. Addressing this issue may require developing new evaluation methods that are less susceptible to training data leakage.