SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Google Trends tokens are IP-locked, silently breaking scrapers behind rotating proxies

0
·1 views

A developer building a Google Trends scraper on the Apify platform discovered that widget tokens issued during the initial API call are tied to the originating IP address. The scraper worked flawlessly on a local machine but failed in production, where proxy servers rotated IP addresses between sequential requests. Counterintuitively, residential proxies — considered higher quality — caused more failures than datacenter proxies because they rotated IPs more aggressively, invalidating tokens even for basic timeline data. The fix involved pinning all requests within a single scraping run to the same IP using a sticky proxy session ID. With this change, a two-term comparison completed in roughly 16 seconds and returned full timeline and related-query data reliably.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingHacker News ·

OpenAI Launches Advertising Platform for ChatGPT

OpenAI has introduced a dedicated advertising platform for ChatGPT, accessible at ads.openai.com. The move signals a significant shift in OpenAI's monetization strategy beyond its existing subscription model. The announcement has drawn notable attention on Hacker News, accumulating 70 points and 60 comments. This development suggests OpenAI is exploring ad-based revenue streams to support the growing operational costs of its AI services.

0 comments Read more at Hacker News
0
ProgrammingDEV Community ·

Developer releases lightweight AI agent library with pause, resume, and human approval

A developer has published yieldagent, an open-source JavaScript library for building AI agents in roughly 200 lines of code with no external dependencies. The library implements a standard agent loop — sending messages to a model, executing tool calls, and repeating until a final answer is returned. A key feature is human-in-the-loop support, allowing specific tool calls to be paused for approval before execution, with resumable state that can be serialized and stored. The library is provider-agnostic, supporting any OpenAI-compatible API including Ollama, and offers optional Zod-based tool validation and streaming support. The project is available on npm and GitHub, with a browser-based demo that requires no API key.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How a Repetitive Work Task Led a Non-Programmer to Learn Coding with AI

A non-technical professional who once considered coding inaccessible began learning Google Apps Script after noticing a time-consuming, repetitive data-entry task at work. Using AI as a starting point, they generated code, debugged errors iteratively, and eventually built a working automation. The process introduced them to core programming concepts such as variables, conditions, and triggers through hands-on problem-solving rather than formal study. Despite the help of AI, they found that understanding the code — not just copying it — was essential for maintaining and adapting their scripts. The experience reshaped their view of coding as a practical, learnable skill for anyone looking to improve everyday workflows, not just professional developers.

0 comments Read more at DEV Community