SShortSingh.
Back to feed

Agents Marking Their Own Work as Done Is Not Verification, Argues Test-Loop Guide

0
·1 views

The article argues that when the same agent writes and approves its output, a 'looks done' judgment reviews nothing and merely confirms the agent's existing intent. Its proposed fix separates roles: an orchestrator drives the process, a builder fixes gaps, a critic script executes real checks and produces a score, and a separate reviewer in a fresh context signs off. Checks are declared up front with commands and weights, and rounds are capped so non-converging loops stop. A fail-closed security gate blocks delivery on secrets or high-severity findings even when tests pass. The piece also warns of pitfalls such as pipes masking exit codes, timeouts silently scoring zero, and loose or regenerated baselines that make failures look like passes.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Anchor Coverage: A CI Gate That Checks Model-Drafted Docs Against Evidence

A DEV Community post describes a CI gate for documentation changes where model-drafted prose must be traceable to machine-readable sources. The author illustrates the problem with a hypothetical docs PR containing a renamed CLI flag that linting never caught. The proposed workflow splits each change into three lanes: script-extracted facts, model-drafted connective prose confined to delimited blocks, and human-owned commitments the model may not touch. A Python script, anchor_gate.py, enforces this by requiring every fact in a draft block to carry a claim anchor resolved against a claims file. The gate fails the build when the share of unanchored paragraphs exceeds a configured budget.

0
ProgrammingDEV Community ·

Argo CD rollback: choosing between git revert and argocd app rollback

A bad image tag merged at 16:50 and auto-sync deployed it, causing 500 errors in the checkout service. Teams facing this must pick between reverting the commit in Git or running an Argo CD rollback directly against the cluster. Both approaches work but differ in how they interact with auto-sync, audit trails, and later fixes. The right choice depends on sync policy, branch protection rules, the scope of affected applications, and whether stateful changes such as migrations are involved. Because sync and history behavior has changed across Argo CD releases, the article advises checking the official documentation for the installed version.

0
ProgrammingDEV Community ·

Company-brain graph outperforms retrieval on HERB held-out company questions

A developer built inherit-brain for a Graph Hacks track on company knowledge, storing each product's records as a FalkorDB graph of people, documents, bugs, and pull requests. An agent answers questions by calling nine read-only tools that run fixed, parameterised Cypher queries rather than writing its own. On a spec question for one product, the graph tools returned all ten authors and reviewers, while retrieval with the same model returned only one. The project was tested on Salesforce's HERB benchmark, a simulated company, using 30 product graphs and 1,203 held-out questions. The author reports that a complex bug-reopening question was answered correctly through a multi-step join, where retrieval named the wrong engineers.

0
ProgrammingDEV Community ·

Developer Launches Agent Colony, a Community Where Only AI Agents Can Post

A developer has built Agent Colony, a platform where AI agents alone can post, comment, claim tasks, and earn reputation, with humans limited to read-only access. The project aims to test whether agents can self-organize and collaborate without human moderation. Agents receive cryptographic Ed25519 identities and earn karma for completed tasks, which can be carried across platforms. The creator reports more than 100 registered agents, over 7,000 messages, and more than 10 open tasks with rewards. The developer says agents can self-organize to some degree but still need help getting started.