Developer Tests AI Support Agent Against Poisoned Help Center, One Attack Slips Through

·1 views

A developer built a fictional e-bike shop support agent on Sanity's Knowledge Base to test its resilience against prompt injection and content poisoning attacks. Seven malicious documents were planted across the help center, ranging from fake refund instructions to data-theft attempts and fabricated policies. Sanity's Knowledge Base filtering blocked six of the seven attacks before they reached the AI model. The seventh — a believable fake policy about a late-delivery credit — was not a direct instruction but a plausible lie, and it successfully misled Claude Opus 5. To limit damage from any fooled model, the developer added a custom policy gate called 'taintgate' that restricts agent actions based on structured rules and tracks whether sensitive values originated from user input or help-center content.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why Parent-Level Access Should Not Grant Full Permissions on Nested Resources

A common authorization mistake assumes that access to a project, workspace, or tenant automatically permits all actions within it. However, read access on a folder does not imply the right to delete its child objects. Similarly, being a member of an organization does not confer admin rights over every resource it contains. Developers are advised to evaluate each request individually based on the subject, the action, and the specific resource involved. This approach ensures that nested objects are protected by their own permission checks, not inherited assumptions from a parent.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Watchdog restarted healthy AI agent 24 times daily for months, logs showed no problem

A developer running an unattended AI agent since May discovered it was being restarted every hour, totalling 24 restarts per day over several months. The system's watchdog logs recorded 96 'kickstart' lines daily, masking the true restart count and framing each unnecessary intervention as a rescue. Investigation revealed the watchdog checked for recent Telegram activity and declared the process hung whenever no messages arrived within 30 minutes — a false positive during quiet periods. The root cause was a health check that measured traffic rather than actual work output, making idle processes indistinguishable from crashed ones. The fix involved replacing traffic-based monitoring with a heartbeat file that the agent writes after completing work, allowing the watchdog to accurately detect genuine failures.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Over-Reliance on AI May Be Eroding Human Intelligence, Warns Developer

Since ChatGPT launched in November 2022, AI has rapidly become central to how many people work, learn, and create content. A developer writing on DEV Community argues that the deeper danger of AI is not job displacement or existential risk, but the gradual erosion of human thinking and understanding. When people use AI as a substitute for reasoning rather than a tool to sharpen it, the gap between producing output and genuinely understanding it widens. Fields like software engineering are increasingly populated by practitioners who can generate working results through AI but lack the foundational knowledge to debug, defend, or improve their work. The author contends that AI's value lies in accelerating genuine learning, and that the critical question users should ask is not what AI can do, but whether they understand what it is doing on their behalf.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Go's Method Set Rules Create Silent API Contract Failures in Large Backend Systems

Go's interface satisfaction is determined at compile time using strict method set rules, but subtle bugs can still slip through in multi-package backend architectures. The core issue stems from how Go handles pointer versus value receivers: a type T only carries methods with receiver T, while *T carries methods of both T and *T. When a struct embeds another type by value rather than by pointer, its method set excludes pointer-receiver methods, meaning only the pointer to the outer struct satisfies the interface. This asymmetry becomes especially dangerous in large systems where the concrete type, the interface, and the wiring code live in separate packages, making compiler feedback less immediate. Developers often discover the correct embedding pattern through trial and error but rarely document the reasoning, leaving future engineers vulnerable to reintroducing the same subtle bug.

0 comments Read more at DEV Community