PULP Engine Uses Zero-Allocation Hot Paths to Handle Millions of Log Events Per Second
A native Windows telemetry engine called PULP has been designed in C11 to process high-volume logs at tens of millions of events per second on standard hardware. The system eliminates dynamic memory allocations on its ingestion hot path entirely, pre-allocating all memory pools at startup to avoid non-deterministic latencies. Incoming structured events are converted into fixed 32-byte cache-aligned binary records, each fitting within half a standard 64-byte L1 CPU cache line to maximize memory bus efficiency. To avoid lock contention during multi-core scaling, each worker thread maintains its own isolated dictionary table using a multi-hash cascade strategy instead of shared global structures. The architecture also forgoes traditional LRU eviction mechanisms, which require access-order tracking, in favor of an evictionless design that preserves throughput.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in