LLMs Miss Critical Security Bugs in Code Reviews, Benchmark Study Finds

·1 views

A developer built a benchmark called 'Plausible PR' to test whether four leading AI models — Claude Sonnet 5, GPT-5.5, Gemini 3.7 Flash, and DeepSeek-R1 — could detect security vulnerabilities hidden inside seemingly routine code refactors without being prompted to look for them. Each of the 10 test pull requests was designed to appear like a harmless cleanup while actually removing a security safeguard such as a permission check, rate limit, or input boundary. Claude Sonnet 5 performed best with 9 out of 10, while GPT-5.5 scored lowest at 6 out of 10. Notably, all four models failed to flag a bug where switching to a string comparison caused crashes on non-ASCII input instead of returning a clean authentication failure. The findings suggest AI code reviewers reliably catch keyword-triggered issues like SQL injection but struggle with subtle edge-case vulnerabilities that require tracing unusual inputs through logic.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How Tailored Engine Logic Makes Programmatic Landing Pages Actually Work

Generic programmatic landing pages often fail users because their underlying mechanics are one-size-fits-all, even when the copy suggests otherwise. A developer expanding a countdown timer tool into verticals like pomodoro, exam, and presentation modes identified two core problems: mismatched engine behavior and visual clutter from unreviewed AI-generated design elements. Domain-specific requirements — such as an overtime counter for presentation speakers or silent audio alerts for exam halls — cannot be solved by simply swapping headlines on a shared template. The proposed solution uses a single unified timer engine driven by a structured schema, where domain-specific hooks activate conditionally based on the active use case. Automated end-to-end testing is recommended to verify that each vertical's unique functional requirements are consistently met.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Semi-Linearizability Model Cuts Coordination Costs While Preserving Data Consistency

Semi-linearizability is a consistency model designed for geo-distributed systems that classifies operations into strong, weak, and semi categories based on their ordering requirements. Unlike traditional linearizability, which applies uniform global coordination to all operations, this approach reserves expensive consensus only for operations that genuinely require it. A prototype system called DeMon implements the model using causal broadcast for fast local operations, targeted consensus for critical ones, and vector-clock-style watermarks to reconcile the two paths. Benchmarks on the RUBiS workload show up to four orders of magnitude improvement in latency for common-case operations. The model is illustrated through an auction system where frequent Bid operations run locally and fast, while the rare CloseAuction operation uses consensus to ensure all relevant bids are correctly observed before a winner is determined.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

LectuLibre Scales Multilingual Book Search to 10,000+ Titles Using PostgreSQL FTS

AI-powered book translation platform LectuLibre faced severe search performance issues as its library grew to over 10,000 books across five languages. Initial SQL ILIKE queries were returning results in over 500ms and failed to handle language-specific features like stemming, stop words, and diacritics. The team evaluated external tools such as Elasticsearch and Meilisearch before opting for PostgreSQL's built-in full-text search using tsvector and tsquery types, avoiding added operational complexity. They designed a separate database table to store one tsvector per language per book, paired with a GIN index to enable fast, language-aware lookups. The approach brought query times down to a target of under 50ms while scaling on their existing Python/FastAPI and PostgreSQL stack without introducing new infrastructure.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

10 Dev Tools Reshaping Media Asset Pipelines for Web Developers in 2026

A curated guide highlights ten developer tools and AI-powered utilities that are transforming how media-rich web applications handle asset management and automation. Tools like Sharp, a Node.js image processing library, offer server-side resizing and format conversion at speeds four to five times faster than traditional alternatives. Other solutions address client-side canvas manipulation, programmatic background removal, real-time video processing via WebAssembly and WebGPU, and SVG optimization. Cloud-based services such as Cloudinary and command-line utilities like FFmpeg are also featured for their roles in dynamic transformation and low-level media encoding. The guide emphasizes that adopting the right combination of these tools can save development teams hundreds of hours across workflows ranging from e-commerce to computer vision dataset preparation.

0 comments Read more at DEV Community