rekuiper 0.500 moves metadata to RAM, hits 200k msg/s on single core at edge

The rekuiper project, a Rust-based reimplementation of LF Edge eKuiper for IoT edge gateways, has released version 0.500-beta with a major architectural overhaul. Earlier benchmarks showed the engine handling 100,000 messages per second on a single CPU core, but performance degraded at 200,000 msg/s, leaving the true throughput ceiling unknown. Profiling revealed the bottleneck was disk I/O — specifically SQLite metadata lookups and flash storage stalls on embedded devices like industrial gateways — rather than CPU or parsing logic. To fix this, developers rebuilt the engine's internal catalog as an in-memory structure, hydrating all stream and rule metadata from SQLite once at startup and serving subsequent reads entirely from RAM with no filesystem calls. Hierarchical benchmarking at 1,000 msg/s resolution then allowed the team to identify the exact single-core physical throughput limits across multiple real-world MQTT workloads, reaching up to 200,000 messages per second.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in