Developer tests three AI visibility tactics — only one delivers verifiable results
A developer building the marketing site for SearchD, an answer engine optimization practice, implemented three widely recommended tactics to make the site more visible to AI assistants: allowing AI crawlers via robots.txt, publishing an llms.txt file, and adding JSON-LD structured data. Testing revealed that a robots.txt allowing crawlers like GPTBot proves nothing on its own, since CDN bot-management rules can block crawlers before they ever read the file. The only reliable check is to simulate each crawler's HTTP request and confirm a 200 status code is returned, as a 403 or 503 makes all other markup work irrelevant. The llms.txt file, a markdown site index intended to help AI models navigate content, is ignored by Google Search and has no confirmed commitment from any major engine to be read. The developer concludes that while crawler access checks provide measurable, actionable data, llms.txt and JSON-LD serve different purposes than commonly claimed and should not be counted as direct citation drivers.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in