Independent Benchmark Finds Jev AI Confident Only When Correct on Agent Risk Tasks

A software developer published a 60-case benchmark on September 19, 2026, testing Jev, an AI model launched by TypeSafe AI on September 15, 2026, on its ability to classify agent tool-call risk into four categories. The benchmark was designed to assess accuracy, latency, and the reliability of Jev's confidence scores — the operationally critical claim from the model's launch. Both jev-latest and jev-preview achieved 91.7% overall accuracy, with perfect scores on clear cases but lower performance on ambiguous ones, run from a residential connection in Portland, OR. The most consistent finding across all runs was that every incorrect answer carried a confidence score below 1.000, meaning the model never expressed full certainty on a wrong answer. The full benchmark harness, tasks, and per-call results are publicly available on GitHub for independent verification and re-running.