Developer builds AI memory system Muninn, misses leaderboard deadline but self-benchmarks results

·1 views

A developer and their AI partner built a hybrid memory retrieval system called Muninn overnight to enter the Agent Memory Leaderboard, which pits systems against competitors from Tencent, Mem0, Cognee, and MemOS. The team missed the submission window and will try again when the next cycle opens in September. Running the benchmark's public pipeline independently on the LoCoMo dataset, Muninn scored an estimated 72.9% in its best configuration, though the developer cautions this is an internal estimate rather than an official result. The same core system, entered as Perpetual Recall on the separate LongMemEval-V2 benchmark, achieved a confirmed submission score of 56.98% accuracy with a query latency of roughly 2.3 seconds. While mid-pack on accuracy, every system that outperformed it required between 27 and 180 seconds per query, compared to under three seconds for Perpetual Recall.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

YouTube Comments Reveal AI Coding Tools Frustrate Users After the Initial Setup

A developer analyzed roughly 22,000 comments across 140 Korean YouTube videos about AI coding tools and found a sharp disconnect between video content and audience concerns. While the videos focused almost entirely on installation, getting started, and building apps, the most-liked comments expressed frustration over high costs, hallucinated outputs, and the burden of verifying AI-generated work. Users noted that catching AI errors often took more time than completing tasks manually, and that incorrect outputs were difficult to identify because they appeared plausible. The analysis also found that mid-sized channels with 10,000–300,000 subscribers surfaced more specific, practical questions than large channels, where comments leaned toward general anxiety about AI's societal impact. The author concluded that while content teaching beginners is abundant, very little addresses the trust and error-verification challenges users face immediately after starting.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How an Unbounded Queue Crashed a 26-Camera AI System at 3 AM

A production AI video pipeline serving 26 cameras collapsed at 2:47 AM when an unbounded asyncio queue ballooned to 14 GB within 11 seconds, crashing the system. The root cause was a combination of uncapped memory queues, redundant frame allocations, and synchronous code blocking the async event loop. Switching from float32 to uint8 quantization and capping queue sizes to hardware-derived limits boosted throughput from 4 fps to 62 fps while cutting peak memory use from 14.2 GB to 3.8 GB. The engineer also offloaded model inference to a thread executor to prevent the event loop from deadlocking under CPU pressure. The key takeaway is that AI systems do not retain visual memory by default, and treating memory as a hardware-constrained resource rather than an afterthought is essential for stable production deployments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Tests Own AI Planning Engine Against Prompt Injection — Architecture Holds

A developer building PlannerCritic, an open-source engine that pairs one LLM to write plans with a second to review them, ran a structured adversarial test to find security weaknesses in the system. Eleven adversarial goals, 21 injection traps, and 35 regression tests were used, including attempts to bypass MFA, exfiltrate customer data, and override the engine via embedded system commands. In each case, the engine escalated and blocked the requests — not by detecting malicious intent, but by evaluating structural flaws like missing preconditions and weak rollback plans. The critic LLM audited plan structure independently, ignoring injected instructions, while deterministic gates processed logic rather than natural language. The developer concluded that the engine's resilience stemmed from its architecture rather than any LLM-level cleverness, though acknowledged that open seams in the system still remain.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AWS Deploys Multi-Agent System to Cut Cloud Migration from Weeks to Minutes

AWS Professional Services has published production data on a deployed multi-agent system that dramatically reduces infrastructure-as-code development time for enterprise cloud migrations. The system chains four specialized AI agents — handling discovery, IaC generation, governance, and post-migration operations — built on Amazon Bedrock AgentCore primitives. Agents communicate through immutable artifacts stored in S3 and state transitions tracked in DynamoDB, avoiding circular dependencies by never calling each other directly. A portfolio governance agent acts as the sole gatekeeper, running cost, security, and compliance checks before any migration can proceed. If templates fail validation, the workflow halts and the IaC generation agent must resolve all issues before the process can continue.

0 comments Read more at DEV Community