Why Agreement Among Multiple AI Models Can Be a Warning Sign in Code Review

A developer who runs eight AI voices to review code reports that the panel is useful, and a ledger of 18 panel calls shows that 17 returned findings. The author argues the benefit comes from disagreement among models rather than consensus, since models with similar errors tend to converge on the same wrong answer. Research cited includes studies finding LLM errors are more than 60 percent correlated and showing that models often abandon correct answers under user pushback. The piece also points to benchmark scoring that rewards guessing over admitting uncertainty, suggesting models are optimised to answer rather than abstain. The takeaway is that a panel is only valuable while its members keep disagreeing.