Why AI coding agents need auditable runs, not just passing test reports

When an AI coding agent reports that tests passed, that report is only as reliable as the execution evidence backing it up, according to a software engineering analysis. The core problem is that agent logs typically record what the agent claimed, not what actually ran, creating a gap between stated outcomes and verifiable facts. The author argues developers should distinguish clearly between requests — what a user asked the agent to do — and results — what was actually executed by the provider or tool. To make agent runs truly auditable, observations should be graded by source strength, from mere tool acceptance up to provider-confirmed metadata bound to a specific invocation. Crucially, all evidence should be tied to an exact commit hash rather than a branch name or conversation summary, since even a perfectly audited run cannot prove the underlying code is correct.