Study Counts How Often AI Review Findings Get Used in Real Work

A developer questioned whether splitting AI work into many roles, including a separate reviewer, is worthwhile. The article contrasts an arXiv paper from an Institute of Science Tokyo team, which uses a multi-role AI setup for autonomous physics experiments with a bit of human oversight, against Anthropic guidance that favors folding subagents into the main loop. The author describes their own system, which keeps just three roles (implementation, research, review) and delegates only when the benefits outweigh handoff costs. They checked this approach against their real work logs, finding that about 60% of review findings were actually used. The piece is an informal, first-person account and has not been peer reviewed.