Five AI models agreed on facts but split on most judgment calls, study finds

Researchers sent the same 24 questions to five AI models from GPT, Claude, Gemini, Grok and Llama families and compared substantive differences in their answers. The models materially agreed on only 8 questions, while at least one divergence appeared on 16, and one model stood alone on 15 questions. Factual questions showed near-unanimity, whereas recommendations, coding defaults and pricing advice split far more often. The authors noted that lone claims were sometimes the most useful insight and sometimes unverified or wrong, and that a single model never flags its own outlier claims. A control test found that one model's verdicts stayed stable across repeated runs, suggesting the disagreement arises between models rather than from sampling noise. The study was conducted by a builder of a multi-model AI app, which the article discloses.