SShortSingh.
Back to feed

Count-Min Sketch: How Probabilistic Data Structures Count Millions of Events in Kilobytes

0
·1 views

The Count-Min Sketch (CMS) is a probabilistic data structure developed by Graham Cormode and S. Muthukrishnan in 2005 to estimate event frequencies in large data streams without storing every individual event. Unlike exact hash maps, whose memory grows with the number of unique keys, CMS uses a fixed-size matrix and multiple hash functions, keeping memory constant regardless of how many distinct elements appear. The algorithm always returns an estimate equal to or greater than the true count, with the error mathematically bounded by user-defined parameters epsilon and delta. Tools such as Redis, Apache Flink, and network anomaly detection systems rely on CMS for high-scale event counting. CMS complements other probabilistic structures: while Bloom filters check existence and HyperLogLog counts distinct elements, CMS specifically answers how many times a given element has occurred.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

DataCheck Research releases auditable Israeli company-registration trends dataset

DataCheck Research has published an open dataset tracking Israeli company-registration and status-change trends, designed to be transparent and reproducible. The dataset separates registration events from status changes, as the two are not interchangeable and conflating them can produce misleading results. Each data series is documented with its source and extraction method, and the combined series is labeled as a net-movement proxy rather than a definitive count of active businesses. The repository includes CSV files, methodology notes, contribution guidelines, and a citation file, and is released under a CC BY 4.0 license. The project aims to demonstrate that public datasets earn credibility when users can clearly trace the data's origins, scope, and limitations.

0
ProgrammingDEV Community ·

Pulse launches AI-verifiable changelog tool with Ed25519 signing and MCP support

A developer has launched Pulse, a changelog platform at pulse.muuc.cn that cryptographically signs every entry using a project's Ed25519 key, allowing both humans and AI agents to verify the authenticity of updates without trusting the platform itself. The tool includes a Markdown editor, branded public pages, RSS and JSON feeds, and an MCP endpoint that lets AI agents in tools like Cursor and Claude Desktop search and retrieve changelog entries. Pulse also offers a drop-in embed widget, email and Slack subscriber notifications with double opt-in, and a gamified Trust Score leaderboard to encourage consistent shipping. The backend is built on Java 17 and Spring Boot 3.3 with a MySQL database, while the frontend uses Vite, React, and TypeScript, with signing logic mirrored across both a Java server and a TypeScript SDK. The platform is free for one project indefinitely, with Pro access available by application, and the creator is seeking community feedback on key design decisions around key custody and the Trust Score formula.

0
ProgrammingDEV Community ·

KindLink: AI Concept Platform That Matches Your Skills to Volunteer Opportunities

A developer has proposed KindLink, an AI-powered platform concept submitted for the DEV Community Weekend Challenge: Generosity Edition. The platform is designed to bridge the gap between wanting to help and knowing how to act by matching users to volunteering opportunities based on their skills, interests, availability, and limitations. Rather than overwhelming users with generic listings, KindLink's AI would analyze individual profiles to suggest highly relevant opportunities, such as pairing a graphic designer with two free weekend hours to creating educational materials for children. The concept emphasizes that generosity extends beyond monetary donations to include time, skills, knowledge, and resources. The developer's stated goal is to reduce friction in the volunteering process so that vague intentions like 'I should help someday' translate into concrete, scheduled action.

0
ProgrammingDEV Community ·

AI Agent's Role Separation Upgrade Backfired, Generating Zero Optimization Proposals

AgentSelfEdit, an open-source tool that rewrites its own system prompts based on execution feedback, released version 0.3.0 featuring separated-role support, allowing different AI models to handle execution, analysis, and judgment tasks independently. The first real-world test assigned Qwen's 30B model as executor and Mistral Small as analyzer across three iterations, but produced zero proposals and zero prompt promotions. Unlike earlier failures where weak models generated poor edits, this run broke at an earlier stage — the analyzer produced an analysis artifact but never generated any proposals. The developer concluded that swapping in a stronger model for the analyzer role does not automatically improve outcomes, since the analyzer must reason over concrete failure traces rather than abstract task identifiers. The finding challenges a common assumption in multi-agent system design that role specialization alone leads to better performance.