SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

AI Models Fix Security Vulnerabilities Correctly Only 26% of the Time, Study Finds

0
·1 views

1Password's security research unit Off-by-1 Labs tested ChatGPT 5.5 and Claude Opus 4.8 by having them generate 6,080 patches for six real-world CVEs, published on August 6, 2026. Only 26% of patches fully resolved the vulnerability without altering application behavior, while over half failed to fix the bug or introduced new issues. The most common failure pattern was a narrow input-validation check that appeared to work and passed tests, but left the underlying vulnerable code intact. Roughly one in twenty patches actively introduced a new security flaw. Researchers found that when models were given correct fix guidance, success rates rose to around two-thirds, but incorrect framing dropped that figure to about one in six, suggesting the models execute the developer's assumptions rather than independently diagnosing the root cause.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Fitz Framework Lets WASM Components Call Server Functions Without HTTP Boilerplate

Fitz, a full-stack web framework, introduces an @rpc decorator that automatically generates both a server-side HTTP endpoint and a client-side fetch stub from a single async function declaration. Developers can call server logic directly from a WebAssembly component as if it were a local function, using standard await syntax, with no hand-written HTTP handlers, fetch calls, or JSON parsing required. The same type definition is compiled for both the server and the WASM client, eliminating type drift between front and back ends. This is the seventh installment in a series documenting Fitz's component model, and the feature draws conceptual parallels to server actions in Next.js, tRPC, and Remix. The framework handles serialization, routing, and async state updates automatically, requiring no external dependencies beyond the built-in HTTP stack.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Fitz Framework Lets WASM Components Call Server Functions Without Manual API Wiring

Fitz, a web framework, introduces an @rpc decorator that allows developers to mark server-side async functions and call them directly from WebAssembly client components as if they were local calls. The compiler automatically generates both the server-side HTTP endpoint (POST /__rpc/function_name) and a client-side fetch stub, eliminating the need to write routes, JSON glue code, or duplicate type definitions. A single shared type defined in the server module is compiled into both a native struct for the backend and a WASM struct for the frontend, preventing type drift between the two sides. The approach draws comparison to existing solutions like Next.js Server Actions, tRPC, and Phoenix, but requires no external dependencies, code generators, or special directives beyond the single decorator. This feature is presented as Part 7 of the ongoing FitzLiveViews series, building on earlier work covering server-rendered and WebSocket-based component compilation.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Open-Source AgentJIT Compiles AI Agent Workflows from 30s LLM Chains to 0.1ms Python

A developer has open-sourced AgentJIT, a just-in-time compiler designed to eliminate latency and cost inefficiencies in AI agent workflows. The tool works by tracing an agent's tool-call sequence on its first run and compiling the resulting execution path into a deterministic Python AST pipeline for all subsequent runs. This approach reduces multi-step LLM chain execution times from up to 45 seconds down to under 0.1 milliseconds, with no token costs on compiled runs. The compiler automatically inserts runtime input guards that trigger a fallback to the original LLM agent if unexpected inputs are detected, preventing crashes or regressions. AgentJIT is available as a self-contained Python library with no mandatory external dependencies and can be applied via simple decorators.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Oracle Autonomous AI Database 26ai Merges AI, Analytics, and Vector Search in One Platform

Oracle's Autonomous AI Database 26ai is a converged database platform designed to handle relational data, JSON, graph, spatial data, and vector embeddings within a single system. The platform aims to eliminate the complexity of managing multiple specialized tools by combining database storage, machine learning, analytics, and application development capabilities together. A key feature is its autonomous operation, which automates routine administrative tasks such as patching, backups, and performance monitoring, reducing the need for dedicated database administrators. Oracle AI Vector Search, another highlight of the platform, enables semantic similarity searches using numerical embeddings rather than exact keyword matching, powering technologies like Retrieval-Augmented Generation and enterprise knowledge search. The platform targets organizations looking to adopt AI solutions without building and maintaining complex multi-technology architectures.

0 comments Read more at DEV Community
AI Models Fix Security Vulnerabilities Correctly Only 26% of the Time, Study Finds · ShortSingh