Headless browser mode, not IP address, was triggering Cloudflare blocks

·1 views

Web monitoring startup IntelDif discovered that Cloudflare's bot challenge was fingerprinting browser mode rather than IP reputation when their crawler began getting blocked on a monitored domain. A simple 2x2 experiment revealed that switching from headless to headed Chromium bypassed the block regardless of whether a residential or datacenter IP was used. The team solved the display requirement for headed Chrome in containerized environments by running a virtual framebuffer (Xvfb), allowing Chrome to behave as if it were on a desktop. Further investigation revealed a two-rung escalation system: Cloudflare first checks for headless fingerprints, then evaluates datacenter ASN reputation, meaning proxies only become relevant after the browser check is cleared. The team concluded that their paid datacenter proxy was actively worsening outcomes by triggering the second rung once the first was resolved, prompting a redesign of their escalation logic.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

DEV Tutorial Shows How to Build Async Web Forms Without Page Reloads

A tutorial published on DEV Community walks developers through building form submission workflows that avoid full-page reloads using JavaScript's native Fetch API and async/await syntax. The guide explains how traditional synchronous form submissions block the main thread, reset client-side state, and degrade user experience on slow mobile connections. The demonstrated implementation disables the submit button during the network request to prevent double-submissions and provides real-time feedback based on server response. Error handling is built in at both the HTTP status and network levels, with the UI reliably reset in a finally block regardless of outcome. The tutorial positions this async approach as a production-ready pattern capable of delivering sub-200ms perceived response times.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Fixes Silent Regex Bug in Formbricks That Crashed Live Surveys

A contributor discovered a runtime bug in Formbricks, an open-source survey platform with over 12,000 GitHub stars, where invalid regex patterns entered by survey creators were accepted without validation. The flaw existed because the schema only checked that the pattern field was a non-empty string, without verifying it was a legitimate regular expression. This meant a malformed pattern like '[invalid' could be saved to the database and only cause a JavaScript SyntaxError when a real user attempted to submit a response, silently crashing the survey. The fix adds two helper functions that wrap RegExp construction in try/catch blocks, rejecting invalid patterns and flags at the schema level before they reach the database. Placing the validation at the schema layer ensures malformed inputs are blocked even if submitted directly via the API, bypassing any frontend checks.

0 comments Read more at DEV Community

ProgrammingHacker News ·

Developer builds canvas-based note app separating short and long-form notes

A developer has launched a canvas-based note-taking and organizer app after an extended period of solo development. The app distinguishes between sticky notes for quick thoughts and A4-style notes for longer documents, displayed on a visual canvas rather than in a linear list. It includes a quick-access mode using local storage for fast note capture and a PDF export feature offering three custom layout styles. File organization is handled visually through a hierarchy of notes, stacks, and labels that mirrors a traditional folder structure. The developer has shared the project on Hacker News seeking community feedback to help guide further development.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

How a Developer Fixed Soft 404 and Duplicate Page Errors in Google Search Console

A developer discovered that member profile pages on their site were flagged by Google Search Console as Soft 404 and Duplicate without user-selected canonical errors, three weeks after publishing a dynamic sitemap. The root cause was that public anonymous profiles were intentionally minimal, hiding personal information from logged-out visitors, making them appear empty or near-identical to Google's crawler. Rather than enriching the pages — which was not permitted by design — the fix involved adding a noindex,follow meta tag and removing the URLs from the sitemap. The developer also noted that most Search Console coverage warnings, such as redirects, crawl budget decisions, and intentional 403 blocks, are benign by design and require no action. The key takeaway is that pages deliberately lacking content for anonymous visitors have no place in a search index built for those same users.

0 comments Read more at DEV Community