How Structured AI Disagreement Revealed Better Answers Than a Simple Vote

A developer running a two-model AI review system encountered a 1–1 split on editorial rulings when a planned third reviewer hit its usage cap, exposing a core flaw: majority voting requires at least three independent voters to be meaningful. Rather than treating the deadlock as a coin flip, the developer resolved disputes by analyzing what both models had discarded, not just what they had chosen. A key insight emerged from requiring each model to record a 'falsification condition' — the scenario under which its own answer would be wrong — which caused one model to inadvertently invalidate its own pick by citing a plot motif central to the work. The exercise also uncovered a manuscript error that multiple prior reviews had missed, demonstrating that even unanimous AI agreement is limited by the scope of questions asked. The author concludes that recording richer structured outputs, rather than adding more voters, is the more reliable fix for AI panel review systems.