Why Your SEO-Optimized Site May Still Be Invisible to AI Answer Engines

·1 views

Websites that rank well on Google may still be absent from AI-generated answers on platforms like ChatGPT and Perplexity, because these tools rely on their own dedicated crawlers — GPTBot, ClaudeBot, and PerplexityBot — which operate independently of Googlebot. A key reason for invisibility is that many sites accidentally block these bots via blanket robots.txt rules added by site generators or AI-protection plugins. Another major issue is client-side rendering: unlike Googlebot, several AI crawlers do not execute JavaScript, meaning content built with React or Vue may simply not exist in the raw HTML they receive. Missing or broken structured data (schema.org JSON-LD) further reduces the likelihood of being cited, as AI retrieval systems rely on it to understand page context. Site owners can audit their exposure by testing raw HTML responses, validating structured data, checking for stale noindex tags, and optionally adding an llms.txt file — a plain-text summary of site content written specifically for language models.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

AWS Lambda timeout raised 6x to 90 minutes for managed instance event-driven functions

AWS has increased the maximum execution timeout for Lambda functions using Event Source Mapping (ESM) on Lambda Managed Instances (LMI) from 15 minutes to 90 minutes, a sixfold improvement. The change addresses longstanding limitations for workloads such as AI inference, media transcoding, financial simulations, and large-scale ETL pipelines that routinely exceed the previous 15-minute cap. The extended timeout applies only to asynchronous and ESM invocations on managed instances; standard on-demand Lambda functions and synchronous invocations via API Gateway or load balancers remain capped at 15 minutes. Supported event sources include SQS, Kinesis, DynamoDB Streams, Amazon MSK, and self-managed Apache Kafka, while Amazon MQ and DocumentDB mappings are still limited to 15 minutes. AWS confirmed there is no additional charge for the new limit, with standard managed instance compute rates continuing to apply.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Regulated AI Apps Need More Than a Vector Database

Developers at Apex Grid, building a regtech tool for Nigerian microfinance banks, have outlined a key architectural distinction between vector databases and data substrates in AI systems. A vector database excels at fast, embedding-based similarity search, making it useful for the retrieval phase of RAG pipelines, while a data substrate provides a versioned, traceable, and auditable layer linking AI decisions to specific data sources. In financial compliance contexts, the ability to trace why an AI flagged a transaction — including the regulation referenced and the data version used — is a regulatory requirement that vector databases alone cannot meet. The team uses a hybrid approach, leveraging vector databases for initial retrieval and a data substrate for governance and auditability. While this adds storage and schema complexity, they argue the tradeoff is justified given the high cost of untraceable AI decisions in Nigeria's regulated microfinance sector.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Fix Ollama Connection Errors in MuleSoft by Using host.docker.internal

Developers using the MuleSoft Inference Connector to connect to a locally running Ollama instance may encounter a connection refused error when setting the base URL to http://localhost:11434/v1. This happens because the MuleSoft runtime runs inside a Docker container, where localhost refers to the container itself rather than the host machine. The fix is to replace localhost with host.docker.internal in the openAICompatibleURL connector configuration, pointing it to http://host.docker.internal:11434/v1 instead. Once corrected, the connector successfully reaches the local Ollama service and returns a valid 200 response with the model's generated output. The solution was demonstrated on Mule 4 Community kernel version 4.12.0.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How a Successful Build Left Almost No Trace in the Permanent Record

A software build completed with all checks passing, yet the append-only logging system rejected the vast majority of its output — 934 lines were dropped and multiple blocks refused. The root cause was a vocabulary mismatch: print statement names had been renamed in code, but the record's fixed list of accepted names was never updated to match. Because the archive uses a closed vocabulary, unrecognized names are silently refused rather than stored, making the data gap invisible until someone attempted to write the record. The author argues this strict refusal was actually valuable, as an open vocabulary would have quietly accumulated typos and renamed duplicates, corrupting historical counts. The fix involved classifying every refused line into a deliberate outcome — either declaring it or explicitly dropping it by name — so that no unadjudicated output remained.

0 comments Read more at DEV Community