Dev Tool Proactive Scan Scores 20/20 on Bug Detection in Internal Benchmark
Software development platform published the full benchmark results for its Proactive Scan tool on Tuesday, revealing a 20-out-of-20 detection rate across single-file and cross-file defects in a 28-case test suite. The benchmark was run on July 2, 2026, against a staging API using Voyage embeddings and Claude Haiku 4.5, with all test cases written fresh in late June to avoid model training-data contamination. Twelve of the cases specifically targeted cross-file bugs — defects invisible to standard linters because the flaw only emerges when two or more files are read together with shared context. The tool also produced zero false positives across eight clean control files. The team acknowledged a conflict of interest since they wrote both the test cases and the grader, but published the full corpus and scoring methodology to allow independent verification.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in