Objective stop signals beat LLM self-judgment in agent loops, small study finds

A software developer published findings comparing two types of stop signals in AI agent loops for verifiable coding tasks. In the experiment, agents using objective red-line criteria — such as passing a test suite — converged successfully in all 9 of 9 trials, while those relying on the LLM's own yes/no self-judgment succeeded in only 2 of 9. The self-judgment condition produced at least four false negatives, meaning the model incorrectly concluded the task was incomplete even when the code was correct. The author cautions that the sample size is just three tasks with three trials each, making the results directional rather than statistically significant. The principle is also explicitly scoped to tasks with objectively verifiable outputs, such as code or schema validation, and does not apply to open-ended creative or analytical work.