Count-Min Sketch: How Probabilistic Data Structures Count Millions of Events in Kilobytes
The Count-Min Sketch (CMS) is a probabilistic data structure developed by Graham Cormode and S. Muthukrishnan in 2005 to estimate event frequencies in large data streams without storing every individual event. Unlike exact hash maps, whose memory grows with the number of unique keys, CMS uses a fixed-size matrix and multiple hash functions, keeping memory constant regardless of how many distinct elements appear. The algorithm always returns an estimate equal to or greater than the true count, with the error mathematically bounded by user-defined parameters epsilon and delta. Tools such as Redis, Apache Flink, and network anomaly detection systems rely on CMS for high-scale event counting. CMS complements other probabilistic structures: while Bloom filters check existence and HyperLogLog counts distinct elements, CMS specifically answers how many times a given element has occurred.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in