SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer builds dual-LLM review system after 157 agent plans reveal planning flaws

0
·2 views

A developer running 157 AI agent plans against a real large language model found that most failures originated in flawed planning, not flawed execution. To address this, they built PlannerCritic, a system that separates plan drafting and plan review across two distinct LLMs to avoid self-reinforcing errors. Deterministic structural checks run first, verifying ordering, rollback coverage, and preconditions without reading goal text, making them resistant to prompt injection. The system operates within a bounded revision loop, and if consensus cannot be reached, it escalates to a human with a single targeted question rather than guessing. A real-world test involving a blockchain chain-split recovery task demonstrated the approach, with the critic flagging multiple sequencing blockers in a plan that had initially appeared sound.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

New Quality Standard Proposes Audit Checklist for LLM Memory Architecture

A versioned audit checklist titled 'Memory Architecture Quality Standard for LLM Assistants' (v1.1, dated August 21, 2026) has been published to help developers evaluate long-term memory systems in AI assistants and agents. The standard covers the full memory lifecycle — from input ingestion and storage to retrieval, context assembly, and feedback loops — across relational databases, vector indexes, caches, and graph stores. Items are categorized by criticality, with blockers flagged for risks such as data poisoning, memory leakage, secret exposure, and uncontrolled memory growth. The checklist requires auditors to map all storage systems and verify read/write ownership before proceeding through sections covering input validation, retrieval logic, and prompt injection risks. Each finding must be backed by concrete evidence such as file references, SQL query results, or prompt snapshots, as unsupported assertions are not considered verified.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Graph Database Benchmark Reveals Memory Limits and Surprising Performance Gaps

A developer benchmarking six graph databases under strict resource constraints — 0.5 vCPU and 256MB RAM — found that Memgraph and ArangoDB both crashed before completing the test. Memgraph failed due to its in-memory transactional engine requiring more memory than the 256MB cap allowed, while ArangoDB miscalculated its cache size by reading the host machine's RAM instead of the Docker container's limit. To keep the benchmark valid, the developer added Kùzu, an embedded graph database, which peaked at just 1.94MB of memory while handling the same dataset. Despite lacking traditional index support, Kùzu outperformed cloud-hosted AuraDB's indexed lookups by roughly 40 times, largely exposing internet latency as the dominant cost in cloud database queries. The findings highlight that no single graph database excels across all workloads, with FalkorDB winning on traversals but underperforming on aggregation tasks.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

A misplaced 'finally' clause silently broke an AI phone agent after every first reply

A developer building an AI receptionist for a real landline discovered that callers were greeted correctly but the agent stopped responding after the first exchange, with no errors or exceptions raised. The WebSocket connection to the speech provider remained healthy throughout, making the bug nearly invisible to standard monitoring. The root cause was a Python 'finally' block inside an async generator that marked the connection as closed whenever a 'break' statement exited the loop at the end of each conversational turn. Since the agent checked the 'is_connected' flag before listening for follow-up audio, it silently refused to process any further input despite the socket being fully operational. The fix involved removing the cleanup logic from the generator itself and delegating connection teardown to a dedicated close method, ensuring a consumer breaking out of iteration no longer mimicked a genuine disconnection.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Next.js 'hostname not configured' error: fix it in next.config.ts, not your CSP

Next.js throws a runtime error when next/image tries to render an image from a remote host not listed in the images.remotePatterns allowlist in next.config.ts. The restriction is intentional: since Next.js fetches source images server-side to optimize them, allowing arbitrary URLs would turn the optimizer into an open proxy. The fix requires adding the correct protocol, hostname, port, and pathname to remotePatterns, then restarting the dev server, which reads the config only once at startup. Optimized images are served from the same origin via /_next/image, so no CSP changes are needed. The older images.domains setting is deprecated since Next.js 14 and slated for removal, making remotePatterns the correct approach going forward.

0 comments Read more at DEV Community