AgentInspect Tool Uses Trace Contracts to Make AI Agent CI Tests More Reliable

·1 views

Developers testing AI agents often face a dilemma: overly strict assertions break on minor output changes, while loose checks can miss dangerous agent behavior like skipped authorization or repeated destructive tool calls. AgentInspect, an open-source TypeScript toolkit, addresses this by introducing deterministic trace contracts that evaluate agent behavior from captured execution logs rather than grading live LLM output. The approach lets teams define rules such as required tools, forbidden tools, call order, token limits, and run status, all verifiable without invoking another LLM as a judge. When a contract check fails, the tool reports specific findings and exits with a non-zero status code, making it suitable as a CI gate. The TraceContract API is currently in beta as of agent-inspect version 6.17.4 and evaluates persisted trace evidence after a run completes, leaving real-time enforcement to the agent application itself.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Launches Free Local-First Productivity App Lexis With Built-In AI

A developer frustrated with data-hungry productivity tools has built Lexis, a free, local-first productivity suite available at lexisapp.xyz. The app combines six tools — habits tracking, notes, journal, tasks, documents, and an AI assistant — into a single platform. Lexis requires no sign-up or subscription, and all user data is stored locally on the device rather than on external servers. The app runs as both a web app and a desktop Electron app, powered by a Next.js and TypeScript stack with IndexedDB for local storage. Its built-in AI assistant, Noor, uses NVIDIA-backed models and is designed to function as a personal tool rather than a corporate chatbot.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How a Simple CircleCI Cache Key Mistake Silently Breaks Your Builds

A common but hard-to-spot bug in CircleCI pipelines can cause builds to run against outdated dependencies without any visible errors or failed jobs. The root cause typically involves three missteps: checksumming the package manifest instead of the lockfile, misunderstanding how restore_keys does prefix matching rather than exact matching, and lacking a manual cache-busting mechanism. Because CircleCI reports a cache hit regardless, the install step may silently reuse stale modules, leading to bugs that appear in production but cannot be reproduced locally. The recommended fix involves checksumming the actual lockfile, ordering restore_keys from most to least specific, using a frozen install command, and including a version prefix in cache keys to allow forced invalidation. Applying these four changes transforms silent cache failures into visible, actionable build errors.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GitHub Project 'follow-builders' Curates AI Builder Content from X and YouTube

A GitHub repository called 'zarazhangrui/follow-builders' is trending after gaining 84 new stars in a single day. The project functions as an AI builders digest, tracking notable creators on X and YouTube podcasts and condensing their content into shorter, easier-to-scan summaries. It aims to solve a common productivity problem for developers: AI research discussions are spread across long videos, fast-moving social feeds, and repetitive announcements. However, the tool carries trade-offs, including potential loss of context in compressed summaries and possible selection bias in which creators are featured. Its recommended use is as a discovery queue to help developers follow AI progress efficiently, rather than as a replacement for primary sources.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Search Features Make SEO Visibility and Attribution Harder to Track

The rise of AI-powered search experiences such as Google's AI Overviews and AI Mode is complicating how businesses measure their online visibility. Unlike traditional SEO, which relies on rankings, clicks, and referral data, AI-generated answers can satisfy users without directing them to a publisher's website. This weakens the link between search exposure and measurable site traffic, making attribution more difficult. Industry observers note that visibility in AI search can also vary depending on how a query is worded, adding another layer of inconsistency. Experts recommend that businesses build a structured, repeatable process to monitor how their content appears in AI-generated answers for their most commercially important queries.

0 comments Read more at DEV Community