Developer builds CacheGuard to block timing attacks on GenAI prompt caches

A developer has created CacheGuard, a Python middleware designed to detect and counter timing side-channel attacks targeting large language model prompt caches. The vulnerability exploits measurable response time differences — roughly 20ms for cache hits versus 200ms or more for misses — allowing attackers to infer whether another user recently submitted an identical prompt, potentially exposing sensitive business data. CacheGuard intercepts LLM requests, calculates risk scores using statistical methods including the Kolmogorov-Smirnov test, and monitors prefix entropy via a trie structure to identify systematic probing. When a threat is detected, the middleware autonomously responds by injecting random delays, bypassing the cache to eliminate timing signals, and isolating the suspected attacker's session. The tool is built on top of SigNoz, an open-source observability platform, which serves as the real-time visualization backbone for all threat metrics and OpenTelemetry data.