How serving markdown to AI agents can cut your site's token costs by two-thirds

When an AI agent visits a webpage to complete a task, it consumes tokens to process the page's content, and heavy HTML loaded with navigation, scripts, and styling inflates that cost significantly. Publishers can reduce this burden through content negotiation, a technique that serves standard HTML to browsers while delivering stripped-down markdown to agents requesting it. On turva.dev, the markdown version of the homepage costs roughly one-third the tokens of its HTML equivalent, and an llms.txt file lets agents map the entire site in a single request. Lighter pages improve accuracy because agents read the full content rather than hitting budget limits midway, reducing the risk of misquoted prices or incorrect terms. The implementation requires only a small piece of edge code that selects the response format based on the request header, leaving the existing site structure unchanged.