Cheap AI code reviewers miss security bugs far more often than frontier models

·1 views

A code review vendor benchmarked a low-cost LLM (Luna, ~$1.20/million tokens) against a frontier model (Astra) across 50 pull requests from projects including Keycloak, Grafana, and Sentry. Luna detected bugs at roughly 20 times lower cost per verified finding, and performed comparably on routine data and logic issues. However, Luna caught only 9 of 24 security bugs versus Astra's 19, and struggled significantly on Keycloak's identity and access management code, where it also produced false positives at a much higher rate. Two critical authorization flaws — a reusable recovery code and a permission override bug — were missed by Luna entirely, as both required reasoning about the broader permission model rather than local code patterns. The study's authors recommend using cheaper models as a triage layer for routine changes while routing authentication and authorization code to more capable models, regardless of per-token cost.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Ollama's 5-Minute Default Caused 214 Model Reloads in a Single Day

A developer running a local AI chat app discovered that Ollama's default 5-minute idle timeout was silently evicting models from VRAM, forcing a costly cold reload from disk on every subsequent request. Over 24 hours and 1,180 requests, the system logged 214 model load events, with cold starts taking 11.4 seconds to first token compared to just 0.9 seconds when the model was already warm. A cron job set to run every 10 minutes was hitting the 5-minute timeout every single time, meaning it had never once used a warm model. The root cause was partly obscured because passing keep_alive in the request body has no effect on Ollama's OpenAI-compatible endpoint — only the native API honors it. Setting the server-side environment variable OLLAMA_KEEP_ALIVE=24h and moving embeddings to a separate CPU-only instance reduced daily model reloads from 214 down to just 9.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Separate Identity, Consent, and Session Management in Patient Portal OAuth

A technical guide published on DEV Community outlines best practices for implementing OAuth-based login in healthcare patient portals using Node.js. The author argues that OAuth handles identity but does not address whether an application should access specific health data categories at a given moment. The piece recommends treating consent and session lifetime as decisions distinct from authentication, with explicit user-facing disclosures showing data category, purpose, and trigger before any access occurs. It also covers refresh token rotation, audit logging of consent state changes, and the risk of revoke actions that only update a UI checkbox without blocking subsequent API calls. Four identity providers — Auth0, Clerk, Keycloak, and Infrai — are compared based on their suitability for portal use cases and their respective trade-offs.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds Auricle, a native Windows music player for YouTube Music

A developer frustrated with the high resource usage of mainstream music apps like Spotify and Apple Music on Windows decided to build his own player called Auricle. The app targets YouTube Music's catalogue and is built using a native UI framework rather than Electron or web-based approaches, with goals of low resource consumption and a clean interface. Key challenges beyond basic playback included stream extraction, queue management, and upstream service compatibility. Distributing an early build revealed a separate hurdle: the unsigned installer triggered antivirus flags, even though scans of the installed executables returned clean results. The project is currently Windows-only, with the developer planning to study its real-world performance before considering a Linux port.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Server-Sent Events Can Replace WebSockets in Next.js Apps

Developers building real-time features in Next.js often default to WebSockets, but Server-Sent Events (SSE) offer a simpler alternative for one-way data streaming. SSE is a browser-native technology that allows servers to push updates to clients over a single HTTP connection. Unlike WebSockets, SSE requires no special protocol and works well for use cases like live feeds, notifications, and progress updates. Next.js supports SSE through its API routes or Route Handlers, making integration relatively straightforward. For applications that only need server-to-client communication, SSE can reduce complexity compared to a full WebSocket implementation.

0 comments Read more at DEV Community