AI Coding Models Doubled Key Benchmarks in 12 Months, Costs Fell 20-Fold

·1 views

Between August 2025 and August 2026, frontier AI coding models underwent a dramatic transformation, with SWE-bench Verified scores rising from 49% to 95% and context windows standardizing at 1 million tokens across all major providers. Anthropic's Claude Fable 5, released in June 2026, leads coding benchmarks at 95% SWE-bench Verified, though access remains restricted and it costs $10 per million input tokens. At the other end of the spectrum, DeepSeek V4 Flash and Google's Gemini 3.5 Flash offer competitive coding performance at as little as $0.14–$0.15 per million tokens, roughly one-seventieth the cost of top-tier models. New benchmarks such as Terminal-Bench, MCP Atlas, and OSWorld emerged to measure agentic and computer-use capabilities that did not exist as formal categories a year prior. Despite these gains, engineers note that code review remains a bottleneck and that models still struggle with high-level architectural decisions, excelling at implementation within established patterns rather than choosing which patterns to apply.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Dev team cuts Quarkus CI pipeline time from 11 minutes to 5 minutes without paid tools

A development team running a Quarkus monorepo on GitHub Actions free-tier runners reduced their CI pipeline duration from 9–11 minutes down to approximately 5 minutes 20 seconds. The gains were achieved without self-hosted runners, larger GitHub runners, or paid caching services. Investigation revealed the bottleneck was not dependency downloads or Maven compilation, but duplicated Quarkus work such as repeated augmentation, redundant application boots, and unnecessary infrastructure startup. The team deliberately avoided splitting integration tests into per-service matrix jobs, as doing so would have multiplied fixed setup costs across each parallel cell. Instead, optimisations focused on eliminating duplicated work within existing jobs while keeping the overall job count small.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Run a Local LLM on Apple Silicon Using MLX or llama.cpp

Mac users with M1 through M4 chips can run large language models entirely on-device using two leading tools: Apple's MLX framework and the community-built llama.cpp engine. MLX is installable as a Python package and can load a 4-bit quantized 3B model in minutes, requiring no API keys or cloud connectivity. The two tools differ in scope — MLX is optimized for Apple's unified memory architecture, while llama.cpp supports a broader range of hardware including Linux and Windows systems. Both frameworks support quantized model formats and offer OpenAI-compatible server modes for application development. Running models locally eliminates per-token costs and keeps user prompts private, with the setup working fully offline.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

npm Supply Chain Attacks Expose Gap in Artifact-Level Dependency Auditing

When a compromised npm package is reported, teams often check their current dependency tree — but this can miss vulnerable versions that existed only briefly during a past build. The Keyv-related package compromise, disclosed by Aikido on August 4, illustrated how a malicious release can spread and be patched before most teams finish assessing their exposure. The core challenge is that modern CI pipelines, lockfile updates, and Docker caching mean a clean scan today says little about what an artifact built yesterday actually contained. Experts argue the useful unit of analysis is not the current repository state but a specific artifact tied to a specific build and its resolved package set. A proposed approach involves accepting lockfiles, build timestamps, artifact digests, and CI logs to reconstruct past builds and classify exposure as confirmed, possible, or unknown — with human review required before any remediation action.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

10 Warning Signs Your Windows PC May Be Compromised and How to Spot Them

Windows users can detect potential security threats by monitoring unusual system activity such as unknown processes in Task Manager, unexpected CPU or RAM spikes, and new startup programs added without user knowledge. Suspicious PowerShell executions, unrecognized USB connections, and sudden changes to Windows Defender settings are also red flags worth investigating. Many legitimate Windows tools like Task Manager and Event Viewer exist for this purpose, but most users do not check them regularly enough to catch threats early. A lightweight monitoring tool called SysPulse has been developed to address this gap by offering real-time process tracking, executable path visibility, and Telegram-based security alerts. The goal is not to replace antivirus software but to give users greater visibility into what is happening on their systems at any given time.

0 comments Read more at DEV Community