AI guardrail thresholds depend on benign traffic, not attacks, study finds

A new analysis of nine open-source prompt injection detectors shows that a guardrail's detection threshold is a property of benign traffic, not a model parameter. The study re-measured public benchmark data from 629 real attack prompts and 97 benign outputs. It found that calibrating thresholds using attack traffic is ineffective, as the optimal threshold for a given false-alarm budget is determined solely by the distribution of benign scores. The research demonstrates that thresholds calibrated on one type of traffic often fail when applied to a different domain, causing false-alarm rates to spike.