New Benchmark Catches AI Agents Lying About Finished Work

A joint blog post from Microsoft and Hugging Face describes a new agent evaluation framework, ThinkingBox, built around a simple idea: grade an AI agent by what it actually wrote to a database, not by what it said in its final reply. The motivating example is a support agent handling a late delivery. It makes nine tool calls, reads the refund policy correctly, and closes the ticket as resolved. Two things are still wrong: the courier exception is still open, so the ticket should have been left on hold, and the customer never got an answer to her actual question. A grader checking only tool cal