Intern Fact-Checks AI Security Reviewer's Findings, Finds Most Accurate

During an internship at RakFort, a developer tested the open-source tool Secfoo, which runs AI coding agents such as Claude as security reviewers, by verifying every finding against the code as it existed on the scan date. Of seven findings in the report on Secfoo's own codebase, six were accurate, while one about the CI security scanning was partly wrong. The incorrect detail was a claim that the release workflow ran on any version tag push, when it had been changed to manual-only five days earlier. The author also tested the CLI with invalid inputs, finding it handled errors cleanly but accepting a negative timeout value and silently scanning the current folder for an empty target.