jamjet-guardrails publishes precision and recall scores for every LLM safety check
A new open-source library called jamjet-guardrails offers nine deterministic checks for LLM input and output validation, with no external runtime dependencies. Unlike most guardrails tools, it publicly reports precision and recall figures for each check against a committed test corpus, with results automatically verified through CI on every code change. Each check returns typed findings with character-level spans, enabling accurate redaction and detailed audit logging rather than just a risk score or boolean flag. The library covers checks for prompt injection, PII, secrets, confusable characters, URL exfiltration, and more, with each rule scoped to specific input or output directions. Notably, weaker scores such as a 0.631 precision on the in-repo PII stress corpus are published openly in the README alongside stronger results, giving developers a transparent view of where the tool falls short.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in