BEAM Benchmark Tests AI Agent Memory at Scale Older Tools Cannot Match

BEAM, the Benchmark for Evaluating Agent Memory, is designed to evaluate how well AI agents retain and update information across long, multi-session conversation histories ranging from 100,000 to 10 million tokens. Unlike simpler recall tests, it spans roughly 100 conversations and around 2,000 targeted questions across ten task categories, making it impossible to solve by merely expanding a model's context window. The benchmark assesses whether an agent can extract relevant facts, update beliefs as information changes, and retrieve correct details after thousands of intervening turns. Older benchmarks such as LoCoMo and LongMemEval are considered nearly saturated, meaning top models score so well that differences between memory systems are hard to detect. BEAM addresses this gap by replicating the scale and complexity that production AI agents actually face when remembering user preferences, project histories, or customer records over time.