Passing Tests Is Not Enough: AI Coding Agents Miss Contextual Correctness

Developers and researchers are raising concerns that AI coding agents, such as OpenAI's Codex and Anthropic's Claude Code, can produce code that passes all tests yet still be architecturally wrong for the current system. As these agents increasingly operate at the repository level rather than generating isolated snippets, the meaning of 'correctness' has expanded beyond functional outputs. Existing benchmarks like SWE-bench evaluate agents on whether their patches satisfy predefined tests, but do not measure how well agents adapt when the underlying architecture or constraints change. Emerging research such as SWE-ContextBench and SWE-Explore is beginning to address context-awareness, but a formal framework for testing context-shift adaptation is still lacking. The proposed solution is not to replace current benchmarks but to add controlled evaluations that measure both an agent's ability to adapt decisions when relevant context changes and its stability when only irrelevant context changes.