SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Why E-commerce Scrapers Should Target Backend Data Before Rendered Pages

0
·8 views

A scraper can receive a successful response yet extract nothing useful, because modern storefronts often send a bare page shell and load prices, reviews, and stock through later JavaScript calls. Browser automation can recover that data, but it adds fingerprinting, CAPTCHA exposure, and slower, more fragile jobs. The recommended approach is to inspect network traffic in DevTools to find the JSON endpoints that supply the data, then test the minimal request needed to reproduce them. Shopify stores are a particular case, since their /products.json catalog endpoint can often be paginated directly without rendering a page. Where such endpoints are absent or depend on tokens, rendering with Playwright remains the fallback.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer localizes a Django app into seven languages, including RTL support

A solo developer translated a large Django application into English plus six other languages: Arabic, Chinese, Spanish, French, Russian, and Bengali. The write-up explains the motivation, which centers on making free project management learning content accessible to non-English speakers and expanding the site's indexable pages. The app's translations for the six non-English languages are AI-assisted drafts, and the app labels them as beta. The article describes handling interface strings with gettext while storing long-form lessons as per-language Markdown files with English fallback. It also covers technical hurdles such as right-to-left layout using CSS logical properties and the language-tag mismatch between zh-hans and zh.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Sloth Brief: a local model turns scattered notes into a short restart checklist

Sloth Brief is an open-source tool that condenses a set of accumulated text notes into a brief, source-linked action list. It runs the Qwen2.5-0.5B-Instruct model locally via Hugging Face Transformers, so no paid API or cloud upload of notes is needed. The model writes one concise summary per note, and Python assigns source numbers from the notes actually supplied, though the author warns that summaries still need checking against the originals. Input files are never edited, and the demo of two synthetic notes passes eight automated tests that cover limits and file handling but not model accuracy. The author notes that no productivity gains have been measured, and that the prototype was built with Codex assistance.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

EU launches AI Act Service Desk offering multilingual expert answers for businesses

The European Commission has opened the AI Act Service Desk and a companion Single Information Platform to help companies understand how the EU AI Act affects them. Businesses can submit questions to experts working with the EU AI Office in any official EU language, after signing in with EU Login. The Service Desk handles specific inquiries, while the Single Information Platform offers self-service tools such as a Compliance Checker and an AI Act Explorer. The platform is currently available in English, French and German, with the rest of the EU languages expected in early 2026. The Commission suggests that companies prepare focused questions tied to particular AI systems and their roles to get the most useful guidance.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Arista warns of exploited SQL injection flaw in VeloCloud Orchestrator, CVE-2026-114625

Arista has published advisory SA-0199 confirming that attackers are already exploiting CVE-2026-114625, a SQL injection flaw in the VeloCloud Orchestrator, the management system for VeloCloud SD-WAN edges, which carries a CVSS score of 9.9. The flaw lets a crafted request alter the SQL statement the application runs, so a low-privilege account such as Enterprise Read Only can perform actions reserved for the operator-level superuser role. Exploitation requires network access to the VCO web interface, which Arista says is exposed by default and cannot be disabled, plus a valid login. Arista has patched its hosted and dedicated orchestrators, and it lists fixed releases for the 5.2, 6.1 and 6.4 trains, noting that the 7.0 train and VeloCloud Gateway and Edge are not affected. Customers on unsupported releases have been directed to Arista's Technical Assistance Center, and the advisory does not say when exploitation began or what attackers did afterward.

0 comments Read more at DEV Community