Tool Uses Statistical Fingerprinting to Verify If AI APIs Serve the Models They Claim

A behavioral fingerprinting method can help determine whether an API endpoint is actually running the AI model it advertises, addressing a gap that model names and valid responses alone cannot fill. The approach, rooted in research showing models have stable one-token response preferences, involves asking simple questions like random numbers or coin flips dozens of times and analyzing the resulting distribution. An open-source TypeScript tool called llm-fingerprint-detector implements this by comparing observed response distributions against trusted references using Jensen-Shannon divergence across multiple task-language combinations. The tool returns one of four verdicts — match, uncertain, mismatch, or insufficient — and researchers caution that a mismatch does not automatically indicate fraud, since model updates, quantization, or load balancing can also shift results. The project, MIT-licensed and hosted on GitHub, includes a public protocol for transparent reporting and recommends repeating tests on separate days before drawing firm conclusions.