SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer Builds AI Vulnerability Triage System, Finds Primary Model Has Systematic Severity Bias

0
·1 views

A solo developer built a five-day autonomous vulnerability triage pipeline that uses two separate AI models — one to assess findings and another, from a different model family (Gemma), to review those assessments. The reviewer model rejected 65% of proposals, flagging 91 out of 140 triage decisions across the observed period. Half of all rejections cited the same issue: the primary triage model consistently inflated severity ratings beyond what the underlying CVSS scores supported. The system also detected prompt-injection text embedded in scanner comment fields and flagged remediation suggestions that lacked specific version details or conflicted with CISA KEV deadlines. The developer noted that a reviewer ratifying most decisions would offer little value, and that the high rejection rate reflected genuine systematic bias rather than a malfunction.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Why Audio Files Should Be Screened Before Being Sent to AI Models

A technically valid audio file can still produce poor AI output if underlying signal problems go undetected before processing. Developer guidance published on DEV Community argues that treating every decodable file as a ready model input turns avoidable input flaws into unexplained AI failures. The recommended approach adds a policy layer between upload and inference, using tools like ffprobe and FFmpeg to inspect metadata and signal characteristics such as level, silence, and channel layout. Checks for issues like near-silent recordings, clipping, or unexpected multichannel layouts allow the pipeline to route problematic files differently rather than processing them blindly. The author also cautions that normalizing low-level audio raises noise alongside the signal and does not substitute for genuinely clean source material.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer fixes four-month Google invisibility caused by blank React pages at index

A solo developer discovered that her personal site, chaotictoejam.com, had been nearly invisible to Google for four months after rebuilding it as a React application. Google Search Console revealed only one indexed page and five total clicks, despite the domain having years of history as a WordPress blog. The root cause was a faulty prerender step that served blank HTML shells for the six most important top-level routes, making them appear as identical empty pages to Googlebot on its first crawl pass. While Google can eventually execute JavaScript to read client-rendered content, this slower second-wave indexing created a duplicate and thin-content problem that hurt rankings. The fix involved adding a static metadata function, per-route structured data, and updating the prerender script to bake real HTML into each top-level route before deployment.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer learns AI coding agents excel locally but miss big-picture context in overbuilt prototype

A software builder developing 'Porch Light', an AI agent that monitors public meeting agendas, found that a simple stack-validation exercise ballooned from a planned two-hour task into a full build day. The developer uses AI tools Kiro and Claude to write and review code respectively, while personally directing and validating the work. A key observation emerged when Kiro wrote a byte-identity test for a folder explicitly tagged as throwaway, and Claude approved it — both agents performed their assigned tasks correctly but neither flagged the broader pointlessness of the work. The developer concluded that AI coding agents reason locally and rigorously without tracking higher-level context like deadlines or throwaway markers, making human oversight of scope and intent essential. The incident prompted a reflection on the value of structured 'spikes' — short, pass-or-fail experiments — as a deliberate mechanism to surface wrong assumptions early before they compound across a project.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How Developer Decisions Shape Brand Visibility in AI-Generated Responses

Generative Engine Optimization (GEO) is increasingly relevant to software development teams, not just marketing departments, as everyday technical choices affect how AI systems represent a brand. Consistent entity naming across schemas, components, documentation, and API references helps large language models build a clearer picture of a product or organization. Structuring content as FAQs, how-to guides, and Q&A blocks with clean markup makes it easier for AI models to accurately reuse a brand's own explanations. Technical fundamentals such as sitemaps, canonical URLs, robots.txt, and an llms.txt file also play a quiet but critical role in how well AI systems can learn from a site. Treating GEO as part of the broader architecture and content strategy — rather than a standalone marketing task — is seen as one of the most effective ways developers can improve how AI talks about their products.

0 comments Read more at DEV Community
Developer Builds AI Vulnerability Triage System, Finds Primary Model Has Systematic Severity Bias · ShortSingh