Why scrapers get blocked before sending a request: TLS fingerprinting explained

·1 views

A developer writing for DEV Community explains that web scrapers are often detected not through HTTP headers but during the TLS handshake, before any request is fully sent. The initial ClientHello message reveals the underlying TLS library being used — Python, Go, and Chrome each produce distinct, identifiable fingerprints. Anti-bot systems use schemes like JA3 and JA4 to hash these handshake parameters and match them against known automation tools. Even matching Chrome's cipher suites is insufficient if the order and GREASE values — random entries browsers insert deliberately — are not replicated exactly. The author recommends libraries like curl_cffi in Python, which impersonate specific browser TLS stacks rather than relying on default system libraries.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Shares Step-by-Step Guide to Pushing First Project to GitHub via SSH

A developer documented their first experience uploading a local project to GitHub using Git and SSH authentication. The process began with configuring Git credentials and generating an ED25519 SSH key, which was then linked to their GitHub account. They created a local folder called 'Kenyan Health Records Analysis' on their desktop, added a README file and a data directory containing a CSV file, and initialized a Git repository. After staging and committing the files, they connected the local repo to a new GitHub repository using an SSH remote link. The completed project was successfully pushed to GitHub and is publicly accessible in their profile.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Pre-Tax vs Post-Tax Tipping: What Engineers Need to Know When Building Checkout Math

Developers building tipping features in checkout or hospitality systems often assume a simple percentage calculation, but the correct base amount varies significantly by region and venue type. In the US and most of Canada, tips are conventionally calculated on the pre-tax subtotal, while in parts of Europe a service charge may already be included in the bill. A recommended approach is to pass both the pre-tax base and tax amount as separate inputs to the calculation function, along with an explicit policy selector, rather than relying on a single opaque subtotal. This makes the tipping logic auditable and easier to adapt across different regional rules or product configurations. Edge cases such as comped items, loyalty credits, and rounding conventions can cause discrepancies between displayed and receipt amounts if the underlying policy is not clearly defined in code.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How developers can adopt an AI-first mindset without overhyping the technology

A software developer shares their perspective on what it genuinely means to be an AI-first developer, arguing that using AI simply for appearances adds no real organizational value. They define an AI-first approach as leveraging AI as a primary tool to meaningfully strengthen what makes an organization relevant and functional. The author emphasizes that understanding an organization's true value requires direct conversations with people, not just AI-generated prompts, since that knowledge is complex and context-dependent. They outline a practical mindset shift for individual developers, noting that the core skill is problem-solving rather than writing code. Key foundations include understanding how generative AI models work, their limitations, and how to properly provide context and choose the right tooling for different environments.

0 comments Read more at DEV Community

ProgrammingHacker News ·

Critic Revisits Wolfram's 'A New Kind of Science' With Sharp Skepticism

A critical review of Stephen Wolfram's book 'A New Kind of Science' has resurfaced on Hacker News, drawing renewed attention. The review, hosted on bactra.org, takes a sharply skeptical stance toward Wolfram's claims and scientific ambitions. It has garnered points and discussion on the platform, suggesting continued interest in evaluating Wolfram's ideas. The original book, published by Wolfram, proposed sweeping new frameworks for understanding science through computational principles. The review's pointed title reflects longstanding academic criticism of Wolfram's self-promotional approach to scientific discourse.

0 comments Read more at Hacker News