Study: Human Oversight of AI Agents Misses 1 in 3 Threats, Developer Turns to Automated Testing

·1 views

A browser-based game simulating AI agent oversight collected data from 40,000 runs and over 409,000 approve/deny decisions, revealing that average players correctly identified threats only 66.3% of the time. Nearly a third of all game sessions ended with a net negative score, highlighting what Anthropic calls 'permission fatigue' — the tendency for human attention to degrade as approval requests accumulate. A senior software engineer at BS23 in Dhaka used these findings to argue that human-in-the-loop review is equally unreliable at the software testing layer, not just at the command-approval level. In response, the engineer restructured the test suite for a Spring Boot AI agent to run 31 automated tests without any calls to a live language model, covering tools, services, vector search, and the web layer. The approach ensures that CI pipelines require no GPU or running model server, with only final end-to-end behavior checks left to manual review.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Builds RAG Pipeline Using Vision Language Model to Handle Mixed Document Formats

A developer has redesigned a semantic retrieval pipeline to address the challenge of processing heterogeneous input formats such as PDFs, spreadsheets, and images. The solution uses a Vision Language Model (VLM) as an extraction layer, converting each document page into an image and sending it to the model via the Ollama API for summarization and keyword extraction. The resulting text summaries are embedded and stored in PostgreSQL using the pgvector extension, enabling semantic search by an AI agent. The pipeline was optimized by switching from the Qwen3.5 20B model to the smaller 4B variant, improving processing speed and reducing resource consumption. Additionally, the model's built-in reasoning mode was disabled after it caused empty outputs on some pages by exhausting the token limit, with conciseness and retrieval quality remaining acceptable after the change.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Tutorial: Build a User-Controlled Operating Mode Layer for AI Voice Companions

A developer tutorial published on DEV Community addresses a core design tension in AI voice companions: users want AI behavior to be predictable and controllable without approving every response. The proposed solution replaces over-reliance on system prompts with an explicit application-owned control layer that enforces one of three user-selected modes — Listen, Explore, or Advise. Built in TypeScript, the architecture places the application between speech recognition and the LLM, validating response types, tracking mode revisions, and managing stale or out-of-order replies. The tutorial integrates with Tencent RTC's Conversational AI platform, which handles real-time voice transport while the application retains ownership of session state, consent, and moderation. Key tools used include Zod for schema validation and a structured proposal system that ensures the LLM's output is checked against active policy before reaching the user.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why pricing pages go stale: the case for observed data over estimates

A developer building a metered LLM product raised a common dilemma: usage allowances on pricing pages are either too abstract to be useful or too specific to stay accurate. The core problem, as analyzed in a post on DEV Community, is one of tense — estimates make forward-looking promises that quietly become false as models, features, and customer behavior evolve. The proposed fix is to replace forecast-style claims with time-stamped observations drawn from real customer data, such as reporting the median usage recorded in a specific recent period. This approach cannot decay because it describes something that already happened, and it gives prospective customers more meaningful context than a raw credit figure or a vague approximation. The author also recommends publishing top-decile usage figures so heavy users can self-select into higher tiers before hitting limits.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How a Single API Key and JSON Schema Can Unify Multi-Provider AI Classification

A software architecture pattern proposes replacing separate OpenAI, Claude, and Gemini client integrations in Node.js apps with a single scoped API key backed by a strict JSON output contract. In this design, a Python classification worker translates incoming requests into chat completions, validates responses, and returns provider-neutral results, while a routing layer maps profiles like 'interactive' or 'batch' to specific models. Application code never receives or selects a provider model ID, keeping the caller isolated from backend routing decisions. The architecture emphasizes that the real value lies not in the shared key itself, but in a reusable eval suite that makes any model route change measurable and reversible. Without logging which model versions produced each result, the approach warns, model routing remains difficult to audit or defend.

0 comments Read more at DEV Community