Developer Replicates Anthropic's Vision-Only Pokémon Claim, But Thinking Logs Raise Questions

A developer independently replicated Anthropic's claim that its Claude Fable 5 model can play Pokémon FireRed using only raw screenshots, building a matching harness that reached the first gym badge in 1,785 turns at a cost of $65.40. The experiment confirmed the core result is real, with full logs, frames, and harness code made publicly available for scrutiny. However, after examining the model's thinking chains, the developer found Claude was recalling memorized game knowledge — such as specific trainer dialogue and city objectives — rather than reasoning purely from visual input. The model had written down plot objectives 141 turns before they appeared on screen and identified off-screen trainer details by memory, suggesting its performance relies heavily on training data rather than vision-based reasoning. This raises questions about what 'vision-only' truly means in this context, as Anthropic published no harness code, step count, cost, or definition of 'minimal' alongside its original announcement.