AgentInspect Tool Lets Developers Audit AI Agent Execution Paths, Not Just Answers

A developer explored AgentInspect, a TypeScript library designed to verify whether AI agents follow correct execution paths — such as actually calling a weather tool rather than skipping it and returning a fabricated answer. The tool records agent traces locally as JSONL files and applies deterministic, rule-based checks on model calls, tool calls, and their order. In a test using Vercel AI SDK with mock language models, two scenarios produced identical final answers, but AgentInspect correctly flagged the run that bypassed the weather tool as a process failure. The experiment also compared AgentInspect with more established tools like Promptfoo, LangSmith, and DeepEval, noting that AgentInspect feels early-stage but is already practical for CI pipeline checks. The author was prompted to try the library after receiving a cold email from its developer, highlighting growing interest in agent observability beyond simple output evaluation.