Yoast and All in One SEO store AI crawler settings in free builds that do nothing

·1 views

An analysis of five free WordPress SEO plugins conducted on September 21, 2026, found that Yoast SEO and All in One SEO include settings to block AI crawlers like GPTBot in their free code, but no code in those free builds actually reads or enforces those settings. Yoast SEO, active on 10 million sites, displays a visible toggle labelled 'Prevent OpenAI GPTBot from crawling,' yet the feature is marked as a Premium preview and has no functional effect in the free version. All in One SEO, installed on 2 million sites, similarly declares AI crawler options in its codebase but never converts them into actual robots.txt rules in its free Lite package. Rank Math and SEOPress do not reference AI crawler names in their free builds at all, though Rank Math does generate an llms.txt file. The author disclosed that Prime SEO, the fifth plugin reviewed, is their own product and is the only free plugin in the comparison that actively writes AI crawler blocking rules to robots.txt.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why Being the Only One Who Knows Your Job Is a Liability, Not an Asset

A reflective piece published on DEV Community explores how individuals and organizations become single points of failure when knowledge lives in one person's head rather than in documented processes. The author attended a session on standard operating procedures and later the i.c.stars iOpener Innovation Conference in Kansas City, where multiple speakers reinforced the same theme. Traci Wills of NAIC warned that the biggest barrier to AI adoption is not access but organizational readiness, including the absence of clear problem statements and trusted data. Craig Moore II of Endeavor Heartland argued that companies lacking consistent, shared SOPs will struggle to automate anything effectively. Dianna Keen of VML Enterprise Solutions added that scaling a flawed system only amplifies its flaws, making documentation a prerequisite rather than an afterthought.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer cuts AI coding agent API hallucinations 20x by adding a read-only research step

A developer running a fully autonomous coding system on a Mac mini found that roughly 38% of task failures were caused by implementation agents inventing or misremembering library APIs they never actually checked. To fix the problem structurally, he introduced a dedicated read-only research sub-agent that investigates APIs and existing code before any implementation begins. The research agent produces a short, structured Markdown brief — including exact method names, version details, and confidence scores — which the orchestrator requires before dispatching work to the coding agent. This two-agent split, where research and implementation tools are strictly separated, reduced hallucinated-API bugs from about one in five tasks to roughly one in forty over three months. The added overhead was less than 10% of total token usage, making the tradeoff highly cost-effective.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Seeks Free LLM APIs with Higher Rate Limits for FastAPI Projects

A developer building a FastAPI backend project is looking for free or generously limited LLM API alternatives to Gemini and Groq. The engineer posted on DEV Community seeking recommendations from peers in the developer community. During active development and testing, the free tiers of both Gemini and Groq were found to hit rate limits very quickly. The developer is specifically interested in platforms or providers that offer higher API call allowances without frequent throttling. They also welcomed general strategies for managing LLM API limits during the development phase.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Cuts AI Workflow Costs Through Four Text-Analysis Trials

A developer ran four trials of a text-analysis AI workflow to study how small design choices affect token usage and cost at scale. The workflow split documents into sentences, tokens, and multi-word terms, then performed multiple classification steps using different AI models in parallel. Early trials with shorter system messages led to poor cache usage and excessive classifications, while more explicit prompts significantly improved efficiency. Removing a single minor instruction — telling the model to respond with just a full stop after completing function calls — visibly impacted workflow behavior, illustrating how trivial-seeming details can have outsized effects. The key takeaway was that simplifying individual AI tasks, rather than maximizing model intelligence, proved more cost-effective as workflow scale increased.

0 comments Read more at DEV Community