Re-run of MCP tool-poisoning scanner benchmark adds two scanners and 14,933 npm/PyPI tool lists

The team behind the WARDEN MCP scanner repeated its earlier comparison, expanding it from three scanners to five and from 986 remote servers to 14,933 npm and PyPI tool lists. All scanners were run in a gVisor container with networking disabled. WARDEN's ruleset update from v10 to v12 produced identical verdicts on every test set, and the two other scanners reproduced their earlier results exactly. The authors also checked that no rules written after v10 had seen the held-out benchmark data, finding no overlap. Several hosted tools were excluded because they send servers to external services for scoring, which prevents an offline comparison.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in