Developer Red-Teams Own LLM Security Gateway, Documents Every Vulnerability Found

A software developer who built a security gateway for large language model traffic conducted four rounds of adversarial testing against their own tool to identify detection gaps. The gateway functions as a transparent proxy that scans requests and responses for leaked secrets, PII, jailbreaks, and prompt injection attempts using deterministic pattern matching in Rust. Testing revealed issues including Unicode tag character smuggling, overly broad keyword rules that blocked legitimate security questions, and narrow base64 detection patterns that attackers could easily sidestep. The developer fixed several vulnerabilities by improving input normalization and anchoring rules to intent rather than isolated keywords, while honestly documenting cases where regex-based detection alone cannot provide a solution. The project highlights a key tradeoff in security tooling: catching attacks without blocking legitimate traffic is as critical as detection itself.