Scan finds big developer platforms cover AI-agent basics but rarely expose machine-readable API discovery
The author built a free scanner that scores websites out of 100 on signals AI agents use, such as llms.txt, robots.txt rules for AI crawlers, server-rendered HTML, and API discovery files. Running it on 22 November 2026 against the homepages of Vercel, Supabase and Cloudflare, scores ranged from 74 to 84. All three already handled the basics like readable HTML, llms.txt, sitemaps, JSON-LD and AI-crawler rules. However, few exposed API discovery files: only one had an MCP config and none had an x402 payment catalog. The author notes the scanner checks only public files and raw HTML, cannot predict whether AI assistants will cite a site, and that the tool is their own product, which also sells a paid setup service.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in