Planning Quality, Not Speed, Drives AI Agent Success, Study of 157 Runs Finds

A six-month study of 157 AI agent deployments across code generation, testing, infrastructure, and data tasks found that planning depth was the strongest predictor of success. Agents that invested three to five times more tokens in upfront planning achieved 4.2 times higher task completion rates and 3.8 times fewer rollback cycles than those optimised for fast execution. These findings have prompted engineers to adopt what the researchers call 'Orca-style' agent fleets, a hierarchical architecture inspired by killer whale social structure where a central planner decomposes goals and specialist executors handle narrow tasks. The model separates strategic reasoning, run on more capable models, from tactical execution handled by smaller, cheaper ones, reducing overall costs by avoiding expensive correction cycles. A shared memory layer and orchestration loop tie the system together, enabling state continuity and replanning when tasks fail validation.