New tool aims to make AI agent binary decisions testable and auditable

·12 views

A developer has identified a core reliability problem in AI agent workflows: when an agent is asked to choose between two options inside a prompt, the decision cannot be logged, tested, or distinguished from a random pick. To address this, they built a dedicated decision tool that isolates the judgment into a separate API call, returning a chosen option, a confidence score, and a reason. The confidence score reflects how far apart the two options were rated, rather than how certain the model sounds about its own answer — a distinction the author argues is critical for building automated gates. Teams can set their own confidence thresholds to either accept a pick automatically or escalate to a human, depending on the cost of a wrong decision. A public MCP server is available for testing the tool, with self-service credentials and machine-readable error codes designed to let agents handle failures programmatically.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Solo dev shares five hard lessons from shipping first iOS app to the App Store

A developer built and launched SnapCal, a barcode-based calorie tracking app for iOS, entirely solo in roughly two weeks using SwiftUI with no third-party dependencies. One key lesson involved mishandling HTTP 404 responses from the Open Food Facts API, which returns that status code when a barcode simply isn't in its database — not when something has gone wrong. Treating "not found" as an error caused users to see alarming server-error messages, and the fix required separating that state from genuine failures in both the network layer and the UI. After submitting the app, the developer received an Apple review request for additional information including a physical-device screen recording, a list of external services used, and documentation of key user flows. The post advises future developers to prepare that review video before submitting, and to distinguish clearly between empty states and actual errors in their app logic.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

ChromaPanel vs React Color Picker Libraries: A Feature and Bundle Size Breakdown

A detailed comparison of four React color picker libraries — ChromaPanel, react-colorful, react-color, and @uiw/react-color — highlights significant differences in features, bundle size, and accessibility support. ChromaPanel offers the broadest feature set, including a gradient editor, image color picker, eyedropper, modern CSS Color 4 support, and contrast tools, all with zero runtime dependencies. In contrast, react-colorful prioritizes minimal bundle size at around 4.9 kB gzipped, while react-color and @uiw/react-color fall in between on both size and functionality. Bundle figures were measured via Bundlephobia on each package's main entry point and may differ from what a tree-shaken production build actually ships. The comparison concludes that no single library suits every project, and the right choice depends on an application's specific color workflow requirements.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Seven Charts That Help Diagnose Why a Key Metric Dropped

When a business metric falls, most dashboards can show that something went wrong but cannot explain why, leaving analysts to rely on manual exports and guesswork. A targeted set of seven chart types — including decomposition trees, Pareto charts, waterfall charts, heatmaps, box plots, scatter plots, and Sankey diagrams — is better suited to root cause analysis than standard reporting visuals. Each chart is designed to answer one specific diagnostic question, such as which segment is dragging a number down or where volume is leaking in a process. The key distinction is that reporting charts confirm what happened, while diagnostic charts help uncover the underlying cause. Choosing the right chart depends on the question being asked, not on the shape of the available data.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Security audit finds critical flaws in Hyperliquid Bridge, up to $1.2B at risk

A security audit of the Hyperliquid Bridge, conducted between October 12 and 26, 2026, identified four critical and six medium-severity vulnerabilities in the protocol's smart contracts. The bridge, which holds approximately $7.5 billion in total value locked across Ethereum mainnet and multiple L2 networks, received an overall risk score of 7 out of 10. Two critical reentrancy flaws were found in the lockTokens and releaseTokens functions, potentially allowing attackers to double-mint or double-release assets via malicious token callbacks. Two additional critical access-control weaknesses were identified, including a single-owner validator set with no multi-signature protection and an upgradeable proxy lacking a timelock mechanism. Auditors warned that these vulnerabilities could be combined with other attack vectors to compromise up to $1.2 billion in assets, and urged immediate remediation before the protocol reaches its next TVL milestone.

0 comments Read more at DEV Community