Developer builds Rust-based web layer Svipall after silent bot-wall bug misled AI agents

·53 views

A developer discovered that AI agents were silently receiving browser challenge pages instead of actual article content, with every layer reporting success despite the wrong page being fetched. The root problem was that no component in the pipeline checked whether the returned page was the one actually requested, causing AI models to summarize bot-wall interstitials as real content. To fix this, the developer built Svipall, a local-first web-fetching layer written in Rust, which classifies blocked responses and attaches structured evidence explaining why a fetch was flagged. Svipall uses a tiered escalation system ranging from bare HTTP requests to stealth browser automation, with measured success rates of 26 out of 31 on public targets but only 3 out of 8 against commercial anti-bot vendors. The developer openly published raw logs including failed attempts, and clarified that the tool is intended for research use and does not bypass authentication or paywalls.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Autonomous AI Agents Are Replacing Static Prompts in Python Development

A DEV Community article argues that carefully engineered prompts and fixed sequential chains are brittle, failing when users provide unexpected input or a step returns malformed output. The author contends that LLM applications are shifting toward autonomous agents that pursue a high-level goal and adapt their plan as they go. Sequential chains are compared to a rigid GPS route, while agents are likened to a self-driving car that can respond to obstacles. The piece promises a hands-on tutorial that builds a self-correcting agent loop in Python, with code drawn from an MIT-licensed ebook and GitHub repository. The article presents this as a broader architectural change in how developers build production AI systems.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why window.onerror Reports Show 'Script error.' and How to Fix Them

A front-end observability guide explains why browser error dashboards often show a generic 'Script error.' with no file, stack, or line number. The root cause is a cross-origin restriction: browsers hide details of scripts loaded from another origin unless they are served with CORS headers and tagged accordingly. The article also notes that window.onerror misses resource load failures and unhandled promise rejections, which fire separate events. It walks through building a tested TypeScript error monitor that captures runtime errors, failed resources, and rejected promises. The author recommends rolling out CORS configuration before adding the crossorigin attribute, to avoid scripts failing to load entirely.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Guide Explains Back-of-the-Envelope Estimation for System Design

A DEV Community article explains how to produce rough traffic, storage, bandwidth, and capacity estimates from an ambiguous system design prompt. The approach is framed as a repeatable sequence: define the workload, compute average and peak requests per second, separate reads from writes, project data growth, and derive network throughput. The author stresses writing down assumptions such as user activity, payload sizes, retention periods, and traffic patterns before doing any arithmetic. The piece also offers quick conversion references, including seconds per day and per year, and notes the difference between decimal and binary data units. Its worked examples are presented as hypothetical inputs rather than benchmarks, and readers are advised to replace them with measured analytics and load-test data.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Store Text Garbled After Analytics Install Due to Late Charset Declaration

On 7 November 2026, a developer added Vercel Analytics to their Astro-built store, and afterward every word on the site rendered as garbled symbols. The source files and build output were correct, so the problem lay elsewhere in how the served page was read. The analytics component injected a script ahead of the charset meta tag, pushing it to byte offset 3,410 in the served HTML. Browsers only scan roughly the first 1,024 bytes for a charset declaration, so they ignored it, and with no charset in the response headers they guessed the wrong encoding. Moving the charset tag to the top of the head, at byte offset 44, restored the text, and the developer now checks how bytes are interpreted before inspecting content.

0 comments Read more at DEV Community