A 94% AI eval pass rate masked a live PII leak across 6 test cases

·2 views

A software team discovered their AI support agent had exposed a customer's account ID and partial billing address to another user, three days after their evaluation dashboard showed a 94% pass rate and cleared the build for deployment. Investigation revealed that 6 of 512 test cases had flagged a PII leak pattern, but these were statistically diluted by a larger volume of minor, low-stakes failures such as tone and formatting issues. The root cause was a flat pass-rate metric that treats all failures equally, regardless of severity — meaning a privacy disclosure counted the same as a verbose response. The team also found their test suite was heavily skewed toward tone and style cases, with roughly 40 such tests for every one probing PII or destructive tool-call behavior. They have since introduced a three-tier severity tagging system and now gate CI deployments on a severity-weighted score rather than a flat pass rate.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Synapse: Open-Source SCA Platform Brings Deterministic, Governed Vulnerability Analysis

KKloudTarus has open-sourced Synapse, a software composition analysis (SCA) control plane designed to unify fragmented security workflows under a single governed system. Built in Go with a clean architecture, it handles SBOM generation across 15+ ecosystems, multi-source vulnerability detection, license classification, and tamper-evident reporting in one place. All scanning and reporting processes are deterministic and reproducible, with no AI model in the core report path. An optional AI layer can only propose findings, requiring a separate human or automated verifier to confirm — the agent cannot validate its own output. Released under the Apache-2.0 license, the project is available on GitHub and welcomes community contributions across areas such as new ecosystem parsers, SAST coverage, and standards like OpenVEX and SARIF.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

JSON, YAML, TOML, or CSV: A Practical Guide to Choosing the Right Data Format

Four data formats — JSON, YAML, TOML, and CSV — each serve distinct purposes and are not interchangeable in software development. JSON is best suited for machine-to-machine communication and web APIs due to its strict, unambiguous structure, while YAML prioritizes human readability for configuration files but is prone to silent errors from misplaced indentation. TOML offers a cleaner alternative to YAML for human-edited configs, with explicit typing and less ambiguity, though it handles deeply nested data less gracefully. CSV remains the go-to format for flat, tabular data such as spreadsheet exports, but cannot represent nested structures. When converting between formats, the key challenge is structural mismatch — particularly when flattening nested JSON into CSV columns or vice versa.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer rebuilt a C Redis clone in Rust to learn the language more effectively

A software developer who had already built a Redis clone in C — including a RESP parser, command table, and append-only file — decided to rebuild the same project in Rust as a learning exercise. Because the design problems were already solved, the developer could focus entirely on how Rust handles implementation rather than what to build. Key differences emerged quickly: Rust's standard library provided data structures like Vec and HashMap out of the box, eliminating hundreds of lines of boilerplate that C required upfront. The developer also found that Rust's enum-based pattern matching enforces exhaustive case handling at compile time, preventing a missing-case bug that had previously shipped undetected in the C version. The developer concluded that rebuilding a familiar project in a new language is a more effective learning method than following tutorials from scratch.

0 comments Read more at DEV Community

ProgrammingHacker News ·

Solo Software Development: Weighing the Key Trade-offs

A blog post by John Jeffers explores the advantages and disadvantages of developing software independently. The article examines what it means to work as a solo developer without a team structure. It was shared on Hacker News, where it received 11 upvotes at the time of aggregation. The piece appears aimed at developers considering or currently working outside traditional team environments.

0 comments Read more at Hacker News