AI Task Completion Checks Are Nearly Random, New Research Finds

A developer discovered two obvious logical errors in an AI-generated output that a second AI had already verified as complete, prompting closer examination of how reliable AI self-assessment really is. Research using AgentProp-Bench found that substring-based AI judgment methods score just 0.049 on the kappa coefficient scale, statistically no better than random chance. Experts attribute this to AI systems comparing surface structure rather than understanding the user's underlying intent, and to 'coherence debt' where longer tasks cause AI to evaluate recent steps without checking consistency with earlier decisions. Mechanical checks with clear standards — such as format compliance or code function calls — remain areas where AI performs reliably and efficiently. However, researchers and practitioners suggest humans should retain oversight for intent-based verification, and recommend prompting AI to identify potential confusion in an output rather than simply asking it to approve the result.