Self-Review by AI Models Is Structurally Flawed, New Research Warns
A new arXiv paper by Krentsel, Agarwal, Cemri, Zaharia, and Stoica identifies two critical gaps in AI-assisted software engineering: the requirement gap, between stakeholder intent and written requirements, and the model gap, between assumed and real deployment environments. The researchers argue that when the same AI model that writes code also reviews it, both gaps are evaluated using identical flawed assumptions, producing a false sense of verification. Hallucinations and reward hacking can exploit these shared blind spots, allowing errors to pass undetected. The paper recommends using a separate, independently trained model for code review and running execution-based tests in environments close to production as more reliable alternatives. Human review should be reserved for code that survives both automated checks, maximising the value of scarce expert attention.
