SShortSingh.
Back to feed

AI Agent Resists Misleading Tool Descriptions in File-Deletion Safety Tests

0
·1 views

A developer conducted four controlled experiments over two months to test whether an AI agent could be manipulated into choosing a dangerous bulk file-deletion tool over a safer, targeted one. In each test, only one variable was changed — including tool docstrings and the number of files — to create increasing pressure toward the riskier action. Even when a docstring contained a false, authoritative tip recommending a pattern that would have deleted a protected file, the model ignored it and reasoned from its own direct observations instead. When bulk deletion seemed more efficient, the agent defaulted to slower but safer repeated single-file deletions rather than adopting a broad, potentially destructive pattern. The experiments suggest that well-designed AI agents can remain grounded in observed context rather than being misled by manipulated tool descriptions.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

How Agentic AI Layers Using LangGraph and MCP Can Strengthen CI/CD Security

Traditional CI/CD pipelines excel at automating builds, tests, and deployments but lack the contextual reasoning needed to assess whether a change is truly safe to deploy. Risky operations like unguarded SQL deletions or removed authorization middleware can slip through when no explicit rule exists to catch them. A proposed hybrid architecture addresses this gap by placing a stateful AI reasoning layer — built with tools like LangGraph, MCP, and A2A — above conventional deterministic scanners such as SQLFluff, CodeQL, and Semgrep. This agentic layer collects evidence from specialized tools and evaluates deployment changes from multiple perspectives before producing an auditable decision. The approach is designed to complement, not replace, existing CI/CD security controls, following guidance from NIST and OWASP on integrating security throughout the software development lifecycle.

0
ProgrammingDEV Community ·

Fluctara Platform Aims to Adapt Audio Entrainment in Real Time Using EEG Feedback

Fluctara is a newly described audio entrainment platform designed to move beyond static binaural beat tracks by dynamically adjusting audio signals based on a user's physiological state. The system uses a 16-layer coupled-oscillator mathematical framework called the Self-Consistent Phenomenological Network to generate binaural, isochronic, and spatial audio computationally rather than from prerecorded loops. When connected to EEG or heart-rate sensors, the platform operates as a closed-loop system, processing real-time physiological data to modify session parameters for each individual. A core feature called the Entrainment Verification Score attempts to measure how closely a user's recorded brainwave activity correlates with the intended target frequency, going beyond simply logging listening time. The platform is positioned as a tool for researchers, developers, and wellness or human-performance organisations seeking a programmable, verifiable signal-generation system.

0
ProgrammingDEV Community ·

Database Migrations: How to Manage Schema Changes Like Code

Database migrations are version-controlled, ordered scripts that evolve a database schema in a traceable, repeatable way, replacing risky manual SQL changes. Tools like EF Core, Flyway, Liquibase, and DbUp track which migrations have run, in what order, and across which environments. Without migrations, teams face environment drift, no audit trail, and no peer-review process for potentially disruptive schema changes. Treating migrations like application code — committed to Git and reviewed via pull requests — ensures dev, staging, and production databases remain consistent. The approach also supports zero-downtime deployments, safe rollback strategies, and seamless integration into CI/CD pipelines.

0
ProgrammingDEV Community ·

Open Design turns any coding agent into a design engine, exports real files

Open Design, an open-source desktop app from nexu-io, delegates all design work to whichever coding agent a user already has installed, such as Claude Code, Codex, Cursor, or Gemini CLI. Rather than using a proprietary canvas or format, it outputs standard files including HTML prototypes, MP4 motion graphics, PPTX decks, and high-resolution images. Teams can encode their brand into a DESIGN.md file that acts as a brand contract, ensuring consistent fonts, spacing, and components across every generated artifact. The app also ships an MCP server and a BYOK proxy, allowing it to connect to over twenty CLI tools or directly to API endpoints from providers like OpenAI, Anthropic, and Google. The project, which has accumulated around 78,000 GitHub stars, recently released version 0.13.0 and introduced a paid cloud service billed by token usage.

AI Agent Resists Misleading Tool Descriptions in File-Deletion Safety Tests · ShortSingh