Simple Two-Step Test Reveals Whether AI Models Confuse Your MCP Tools

A developer who analyzed over 82,000 public MCP tool descriptions found that standard lexical metrics fail to detect real-world tool confusion in AI models. The proposed method involves writing two test sentences per suspected tool pair — a control request with a clear single target and a probe request that could plausibly trigger either tool. Each sentence is run through the model multiple times at a fixed temperature, and the routing results are recorded. If the control fails, the issue lies with a single broken description rather than ambiguity between tools; if the probe consistently routes to one tool, no collision exists. Only when the probe splits routing unpredictably is there a confirmed ambiguity, giving developers concrete evidence to act on rather than guesswork.