Developer builds 7ms prompt-injection detector on 78K samples, skips LLM approach

A developer built a prompt-injection detection system trained on over 78,000 attack samples, deliberately avoiding the increasingly common approach of using a large language model as the judge. Instead, the system relies on deterministic regex and classical machine learning — specifically TF-IDF character n-grams fed into logistic regression — completing each check in roughly 7 milliseconds at no per-call cost. On held-out public benchmarks, the detector achieved 0.895 recall on real-world jailbreaks and 0.799 on obfuscated attacks, both at 1.00 precision, but scored only 0.324 recall on subtle roleplay-framed jailbreaks — a gap the developer openly acknowledges. The developer also highlighted that false-positive rates vary significantly by traffic type, ranging from roughly 0.4% on curated benign input to 5% on open conversational logs, warning that single-number FPR claims from vendors are often best-case figures. The core argument is that a cheap, fast, deterministic layer should handle all traffic, with heavier models reserved only for ambiguous edge cases that justify the added latency and cost.