rekuiper 0.500 moves metadata to RAM, hits 200k msg/s on single core at edge

·48 views

The rekuiper project, a Rust-based reimplementation of LF Edge eKuiper for IoT edge gateways, has released version 0.500-beta with a major architectural overhaul. Earlier benchmarks showed the engine handling 100,000 messages per second on a single CPU core, but performance degraded at 200,000 msg/s, leaving the true throughput ceiling unknown. Profiling revealed the bottleneck was disk I/O — specifically SQLite metadata lookups and flash storage stalls on embedded devices like industrial gateways — rather than CPU or parsing logic. To fix this, developers rebuilt the engine's internal catalog as an in-memory structure, hydrating all stream and rule metadata from SQLite once at startup and serving subsequent reads entirely from RAM with no filesystem calls. Hierarchical benchmarking at 1,000 msg/s resolution then allowed the team to identify the exact single-core physical throughput limits across multiple real-world MQTT workloads, reaching up to 200,000 messages per second.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

DEV Seeks Full-Time Community Manager to Steward Its Developer Community

DEV has reshaped an earlier Community Program Manager opening into a full-time Community Manager role focused on community stewardship rather than program management. The position centers on daily presence on the platform, including publishing, engaging with members, and maintaining a welcoming, high-quality environment. Responsibilities include moderation, managing community support channels, overseeing brand presences such as the MLH organization, and curating content through roundups and newsletters. Candidates are expected to be established DEV members with experience running Discord communities, and the package includes a competitive salary, stock options, health and retirement benefits, unlimited PTO, and travel opportunities.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

osFoundry's Memory and Knowledge Base keep AI-learned facts and return them as context

osFoundry bundles Notes, Knowledge Base, Memory, Lessons, Chat History and Indexed Folders to stop coding agents from forgetting project details between sessions. In the desktop app, each store has its own screen, and a toggle on each controls whether the AI may read it. The Memory screen shows what each agent knows, such as database versions and test commands, and separately lists lessons written only after a reviewer rejected work or a step failed. Users can export or import memories and lessons to a file, since they stay on the device and do not move automatically between machines. A user can also index a repository locally with on-device embeddings, which lets the agent search code by meaning and return exact file paths and line ranges.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Building Audit Logs in Phoenix Without Exposing Passwords

A developer building audit logging for a Phoenix application found that a basic Ecto-based approach had several flaws. Application-level logging can omit sensitive fields like passwords by filtering virtual fields and associations, but it misses changes made outside the changeset path, such as bulk updates or manual database edits. To capture every modification regardless of how it happened, the author turns to PostgreSQL row-level triggers, which fire on every insert, update, and delete. The trigger approach can record old and new values while excluding secrets such as hashed passwords. The article argues that database-level capture is necessary for a reliable audit trail.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why an 89% hallucination rate needs context: what the benchmark actually measures

A developer published a video walkthrough to explain the 89% hallucination figure, which is often cited as proof that AI models fabricate information constantly. The figure comes from a specific evaluation setup, including a defined prompt set, sampling method, and a scoring rubric that counts confident fabrication rather than hedging. The author notes that two runs with slightly different random seeds shifted the result by several points. The rubric distinguishes unsupported claims from factually wrong ones, which means a correct answer that the evaluation cannot verify can still be scored as a hit. The author also describes the cost of running the evaluation, including about 40 minutes of compute, hundreds of thousands of judging tokens, and substantial manual human review to check the rubric.

0 comments Read more at DEV Community