SShortSingh.
Back to feed

jamjet-guardrails publishes precision and recall scores for every LLM safety check

0
·1 views

A new open-source library called jamjet-guardrails offers nine deterministic checks for LLM input and output validation, with no external runtime dependencies. Unlike most guardrails tools, it publicly reports precision and recall figures for each check against a committed test corpus, with results automatically verified through CI on every code change. Each check returns typed findings with character-level spans, enabling accurate redaction and detailed audit logging rather than just a risk score or boolean flag. The library covers checks for prompt injection, PII, secrets, confusable characters, URL exfiltration, and more, with each rule scoped to specific input or output directions. Notably, weaker scores such as a 0.631 precision on the in-repo PII stress corpus are published openly in the README alongside stronger results, giving developers a transparent view of where the tool falls short.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

OpenAI Launches GPT-6 Astra With Multi-Platform Rollout Across ChatGPT, API, and Cloud

OpenAI has introduced GPT-6 Astra, its latest AI model, initially rolling it out to a limited set of organizations before broader availability. The model is set to reach ChatGPT Plus, Pro, Business, and Enterprise subscribers, as well as developers via the OpenAI API and cloud platforms including Microsoft Azure and AWS Bedrock. Astra is designed for multi-step, computer-use tasks such as form filling, CRM updates, coding, and research, with OpenAI highlighting improved accuracy and stronger alignment in sequential workflows. API pricing is set at $10 per million input tokens and $50 per million output tokens, with a faster, higher-cost mode also available. Astra will be included within existing subscription allowances, and workspace administrators will have the ability to enable it for their teams.

0
ProgrammingDEV Community ·

UIPKGE launches 62 free, MIT-licensed chart components for React, Vue, Next.js, and Nuxt

UIPKGE has released a collection of 62 open-source chart components available in separate React/Next.js and Vue/Nuxt libraries. The charts span eight families — including cartesian, circular, hierarchy, flow, distribution, specialty, maps, and a custom wrapper — totalling 62 components across both catalogs. Most components are built on Apache ECharts, while simpler visualizations like progress rings and waffle charts use plain SVG or markup. Following a shadcn-style registry workflow, component source code installs directly into a project, allowing developers to inspect and modify implementations as needed. The collection is free under the MIT license and supports theming via CSS tokens to help charts match an application's existing UI.

0
ProgrammingDEV Community ·

How Google Apps Script Can Automate Repetitive Spreadsheet and Gmail Tasks

A developer discovered that Google Apps Script, a JavaScript-based platform, could automate tedious daily tasks like checking spreadsheets and sending email reminders without needing a backend server. The tool integrates with Google Workspace services including Sheets, Gmail, Drive, Calendar, and Docs. As projects grew more complex, debugging became time-consuming, often requiring repetitive copy-pasting between the script editor and external AI chatbots. The developer later adopted Google Apps Script Copilot, an AI assistant built directly into the Apps Script environment, which reduced context-switching by allowing project-aware queries. The tool was used as a development aid rather than a hands-off code generator, with the developer continuing to review and understand all suggested changes.

jamjet-guardrails publishes precision and recall scores for every LLM safety check · ShortSingh