Bayesian Agent Stop Rule Offers No Edge Over Simple Threshold, Audit Finds

A developer re-examined a June 2026 benchmark that claimed a Bayesian adaptive stop rule for AI coding agents used 41% fewer steps than a fixed budget while maintaining near-perfect task quality. The replication confirmed the original numbers but revealed the adaptive policy performs identically to a plain score-threshold rule, because that simpler rule runs first in the decision stack and always triggers before the Bayesian logic. Further analysis showed both score-dependent strategies break down under noisy judge outputs: at moderate noise levels, the loop falsely believed it had met the quality bar in 99.5% of runs, while actually achieving it in only 71%. A fixed six-step budget, which ignores scores entirely, held true success at 93% regardless of noise. A simple one-line fix requiring two consecutive passing scores restored true success to 97% at the cost of just 1.4 additional steps on average.