ProvGuard blocks prompt-injection by tracking data origin instead of judging text
A developer found that text-classifier defences against prompt injection in AI agents generalise poorly, with a fine-tuned transformer's F1 falling sharply when tested on a different dataset. Attackers can always write text unlike what detectors were trained on, so the approach shifts focus. The proposed system, ProvGuard, labels inputs as trusted or untrusted and requires sensitive tool arguments, such as email recipients or payment details, to originate from trusted sources. Policies are declared in a short YAML file and can require confirmation or block actions once untrusted content has been read. The system was evaluated on the AgentDojo agent-security benchmark, including a model-free replay of reference solutions.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in