HyperLogLog: How a 12KB Algorithm Can Count 100 Billion Unique Items

·5 views

Counting billions of unique visitors with a hash set is impractical because memory usage grows linearly, potentially demanding terabytes of RAM. A probabilistic approach called HyperLogLog, rooted in Flajolet and Martin's 1985 paper and refined in a 2007 follow-up, solves this by trading exact accuracy for extreme memory efficiency. The algorithm works by hashing each incoming ID and tracking the longest streak of leading zero bits seen, using that maximum as a statistical estimator of unique item count. A single such counter is highly volatile on its own, but HyperLogLog combines many counters and averages their results to dramatically reduce error. The result is a data structure that can estimate cardinality across hundreds of billions of items while fitting comfortably within a few kilobytes of memory.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Von der Leyen Warns Frontier AI Could Enable Unprecedented Cyberattacks

European Commission President Ursula von der Leyen used her 2026 State of the Union address to highlight the cybersecurity dangers posed by advanced AI models, warning they could enable hacking at a scale previously unimaginable. She cautioned that such powerful models could fall into the hands of hostile actors, signaling that AI safety, model security, and supplier accountability will remain central to EU policy. Her remarks did not introduce new legal requirements but reinforced the EU's existing governance direction, including the AI Act framework. The EU is also expected to release a dedicated AI cybersecurity plan in July 2026, addressing both the threats and defensive opportunities that advanced AI presents. Von der Leyen additionally called for international cooperation on model evaluation and early warning systems, with partners including Canada and the UK.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Build a Free Client Portal Using Google Sheets and Sheetrocket in 30 Minutes

Agencies and freelancers often struggle with scattered email threads and disorganized project updates, making it hard for clients to track progress. A tutorial published on DEV Community outlines how to build a structured client portal using Google Sheets and a free tool called Sheetrocket, requiring no coding or database setup. The portal includes a dashboard, milestone tracker, invoice history, document links, and a messaging form, all accessible to clients via a single private link. Sheetrocket offers ready-made templates for various business types, including agency, freelancer, and coaching portals, which can be connected to a copied Google Sheet template. The entire setup is estimated to take around 30 minutes, with the business owner retaining full control over the data through their own Google account.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer replaces free hit counter API with Firebase after mobile blocking issues

A developer building a static web app called Daily Doodle attempted to add a simple visitor hit counter using a free third-party API, countapi.mileshilliard.com. The counter worked correctly on desktop browsers but consistently failed to update on mobile devices, with no visible error shown to users. Investigation revealed the likely cause was the API's domain being blocked at the network level by ad blockers or mobile carrier filters. Switching to Firebase Realtime Database resolved the issue, as Google's infrastructure domain is rarely blocked by such filters. The developer shared a step-by-step setup using Firebase's REST API with server-side atomic increments, requiring no SDK or authentication under the free Spark plan.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How One Dev Built a Cheat-Resistant Browser Game Leaderboard with Cloudflare D1

A developer built a browser game called 195-0 Game using Astro, Preact, and Cloudflare Workers, where players draft five people for cabinet roles and run a fictional campaign against 195 nations. A key engineering challenge was making the public leaderboard trustworthy without relying solely on client-reported scores. The solution uses signed draft tokens and server-side move replay: the server reconstructs the game from a seed and action log, then issues a signed result token before any leaderboard submission is accepted. Most content pages are prerendered, keeping the site fast, while the server handles draft issuance, score verification, and shareable PNG result card generation. The developer acknowledges the system validates legal move sequences but cannot confirm human participation, meaning a bot could still submit high scores through valid gameplay paths.

0 comments Read more at DEV Community