Designing a Progress-Aware Controller to Decide When AI Agents Should Stop

A developer argues that deciding when an AI agent should stop is a system-level design problem rather than a matter of prompting or a fixed step cap. Fixed limits like max_steps are useful as safety boundaries but cannot tell whether a task is actually complete, since simple tasks and complex ones need very different numbers of turns. The proposed architecture separates the agent from a controller that checks actions against a budget gate, evaluates progress after each tool call, and tracks completion, progress, and remaining budget. The controller would return one of four outcomes: finish, continue, replan, or stop as incomplete, keeping budget exhaustion distinct from genuine success. The author favors a transparent, rule-based policy for an MVP, durable state storage, and head-to-head testing of fixed-limit, stagnation-aware, and progress-aware policies on success rate, cost, and premature stops.