Developer warns AI 'verification' is unreliable after flawed code copy goes undetected

·2 views

A developer building a second app attempted to replicate a working podcast production pipeline by asking an AI assistant to copy it, then used a separate AI session to verify the result. The fresh session had no context of the original pipeline and confirmed the code looked correct, despite significant production flaws in timing, music fading, and voice settings. The errors only surfaced when the developer ran a real transcript through the new pipeline. The author argues the problem is not blind trust in AI, but mistaking proxy signals — like passing tests or an agent's 'done' — for genuine verification of quality. The experience forms the basis of a new book the developer is writing on how to properly validate work produced by AI coding agents.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Independent audit finds IBM quantum chemistry benchmarks converged to wrong spin states

A researcher has published an independent audit of IBM's iron–sulfur cluster quantum chemistry results, originally presented in a 2025 Science Advances paper as evidence of practical quantum utility. The audit finds that key calculations converged to incorrect electronic spin states, with measured spin expectation values of 4.7 to 7.0 against a target of zero for a singlet state. When IBM's own archived raw hardware data was run through their own pipeline, the larger iron–sulfur cluster converged to a spin-pure triplet state sitting 1,438 millihartrees from IBM's reference energy. IBM's built-in spin mitigation tool was found to expand the determinant subspace fourfold without actually producing the target singlet state, and a related issue filed on the official Qiskit tracker was closed without the core discrepancy being contested. The audit, which was conducted with AI assistance over roughly 72 hours, includes a full reproduction map linking every claim to verifiable files and commands.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Cognous Launches Open-Source Stack to Govern and Audit AI Agent Actions

As organizations increasingly rely on AI agents to perform real-world tasks like sending emails and querying databases, the lack of oversight creates significant operational and reputational risks. Cognous has released the Open Control Stack, a fully open-source framework designed to govern AI agent behavior before actions are executed, not after. The stack consists of four components — Declare, Control, Replay, and Evidence — covering three core capabilities: authorizing what an agent can do, enforcing those rules in real time, and producing auditable records on demand. Unlike traditional application logs that only reveal what happened after the fact, the stack sits in front of agents to proactively allow, block, or escalate actions based on pre-defined rules. The initiative aims to give teams granular, action-level control over AI agents rather than applying a single blanket policy across all agent behavior.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Amazon OpenSearch Ingestion Automates Real-Time Database Sync Without Custom Pipelines

Keeping a transactional database and a search engine in sync has long been one of the harder engineering problems, typically requiring fragile custom pipelines or dual-write logic prone to failure. Amazon OpenSearch Ingestion now offers native change data capture (CDC) integrations with Amazon Aurora, Amazon RDS, and Amazon DynamoDB, eliminating the need for hand-built sync infrastructure. The service handles near-real-time synchronization automatically, including the initial historical data backfill that has traditionally required a separate mechanism. Common approaches like dual writes and polling each carry significant drawbacks at scale, from inconsistent state under load to resource waste and stale results. OpenSearch Ingestion aims to resolve these issues out of the box, letting teams combine strong transactional storage with language-aware, vector-capable search without managing the sync layer themselves.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Moksha Dev Game Adds English-Hindi Toggle With Classical Sanskrit Term Preservation

An indie developer has released version 0.0.9 of Moksha, a philosophy-based game previously limited to Hindi-only players, adding a bilingual English-Hindi language toggle. The update introduces a dedicated i18n.js language manager that stores translation keys rather than resolved strings, allowing in-game alerts to re-render correctly in whichever language is active at the time. Classical Sanskrit terms such as Prārabdha, Puṇya, and Kṛpā were deliberately left untranslated or kept in transliterated form, as the developer argued that replacing them with phrases like 'bad karma' or 'grace' would distort their original philosophical meaning. Vedic verses are retained in Devanagari script even in English mode, with only an explanatory translation appended beneath them, following a principle of omitting no word and adding no unintended meaning. The refactoring process also surfaced several pre-existing bugs, including a typo in an HTML element ID that had silently disabled the toggle entirely and an accessibility error that had prevented screen readers from announcing language changes.

0 comments Read more at DEV Community