Wrong API endpoint, not a block: how a YouTube scraping error cost three months

·15 views

A development team spent three months believing YouTube blocked their datacenter IPs after a probe of the internal API returned zero comments, which they mistakenly interpreted as a block. The real cause was a flawed testing method: YouTube's /youtubei/v1/next endpoint never includes comments in its initial watch payload, as comments require a second request using a continuation token. The team's single-step probe consistently returned empty comment data across all IP tiers — direct, datacenter, and residential — making the false negative appear highly credible. An outdated internal note about IP-class blocking on a different endpoint had been incorrectly generalised to cover all YouTube API surfaces, suppressing further investigation. Once the correct two-step process was applied, all three IP tiers successfully returned comment data, revealing no block existed and exposing one of the largest unserved scraping demand clusters the team had encountered.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

GitHub's new star history endpoint powers a README star chart generator

GitHub restricted its stargazers listing to repo admins and collaborators this year, which broke existing star history tools. In September it added a star history endpoint that returns stars grouped by calendar week for any public repo without authentication. The author built a README chart generator on it, fetching pages in parallel and summing weekly totals, which matched live star counts on seven tested repos. Because README images cannot run scripts, the chart is a server-rendered Svelte component producing an animated SVG, while a separate plain SVG converted to PNG handles social previews.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Word Counting Is Harder Than a Simple Space Split Suggests

A common naive word counter splits text on whitespace, which breaks on contractions, hyphens, em dashes, decimals, and emoji. The article argues this is a text segmentation problem rather than a rounding issue. It recommends the browser's Intl.Segmenter API, using word granularity and filtering by isWordLike to exclude emoji from word counts. It also notes that text.length counts UTF-16 code units rather than visible characters, so grapheme granularity is better for character counts. The author discloses that they built a browser-based word counter that processes text locally without uploading it.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Shell Redirection and Pipes Work: >, >>, 2> and |

The article explains that every command has three text channels: stdin for input, stdout for normal results, and stderr for error messages. The > operator sends stdout to a file and overwrites it, while >> appends to the end of an existing file. Because stderr is stream 2, the 2> operator captures error messages separately, and 2>&1 merges errors into the same destination as normal output, with the order of redirections mattering. A pipe (|) passes one command's stdout directly into the next command's stdin, allowing small tools to be chained together. The piece notes that pipes carry only stdout, so errors from the left-hand command bypass the pipe.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer details AI-assisted plan for an SMS suite in the CabloyJS framework

A developer building an SMS suite for the CabloyJS fullstack framework shared the prompt they wrote and the development plan Claude Code produced in response. The suite is split into four modules: shared SMS infrastructure, SMS verification codes, phone-based registration and login, and a mock provider for development. The plan stresses that infrastructure, verification, and authentication should stay decoupled, and that the mock provider should not alter the real business flow. It proposes reusing the existing captcha and authentication systems, with backward-compatible extensions for phone binding and atomic code consumption. The developer presents the exchange as a representative example of a practical AI coding workflow.

0 comments Read more at DEV Community