Study finds popular PII redaction tools miss nearly all data hidden in AI tool-call arguments

A developer benchmark has revealed that widely used PII redaction tools, including LiteLLM's Presidio guardrail and LLM Guard, fail to scrub personal data embedded in tool-call arguments generated by AI agents, leaking 99.1% of such data in tests. The problem stems from these tools scanning only plain message text, while agentic AI systems also pass sensitive user data inside structured JSON fields like tool_calls[].function.arguments and multimodal content parts. The benchmark tested 120 real documents containing 458 labeled PII items across four languages, drawn from the open-source AI4Privacy dataset. A structured-aware alternative called PrivAiTe, using a combination of Presidio and a locally run ONNX privacy model, reduced tool-call PII leakage to 15.5% in its default preset. The author cautions that even this approach is best-effort defense-in-depth rather than a guaranteed compliance solution.