Experienced developers were 19% slower with AI tools, study finds

·2 views

A 2025 randomized controlled trial by METR found that experienced open-source developers completed coding tasks 19% slower when using AI assistance compared to working without it. The study involved seasoned developers working on their own familiar repositories, making the slowdown particularly notable on codebases they knew best. Before and after each task, participants consistently believed AI had made them faster — estimating gains of around 20-24% — revealing a significant gap between perception and measured reality. Researchers attributed the slowdown to the overhead of prompting, reviewing, and correcting AI output, which added time on top of work developers could already perform quickly from memory. The study also noted that developers who avoided the productivity loss were those who treated AI tool effectiveness as a measurable question rather than an assumption, tracking their own performance rather than relying on intuition.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How Modern Developer Portals Are Rethinking Documentation Search

Technical documentation search presents unique challenges because developers query for specific error codes, API endpoints, and configuration syntax that standard search engines often mishandle. Traditional lexical search tools strip punctuation and split underscores, causing searches like 'wp_insert_post()' to return zero results. Hierarchical page structures further complicate retrieval, as naive engines return entire pages instead of pointing users to the relevant section. To address this, modern portals are adopting hybrid search pipelines that combine lexical matching for structural queries with semantic vector search for conceptual ones. Managed solutions like Algolia DocSearch, alongside self-hosted and AI-powered alternatives, offer varying trade-offs in cost, accuracy, and computational overhead.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Open-Source SIEM to Fix Security Blind Spots Left by Fail2ban

A developer has created CNSL (Correlated Network Security Layer), a self-hosted, lightweight SIEM tool designed to address a core weakness in conventional Linux server security. Standard tools like Fail2ban, NGINX, and database loggers each monitor only their own logs in isolation, making it easy for attackers to slip through by spreading low-volume probes across multiple services. CNSL counters this by correlating activity from web, SSH, database, and cloud logs simultaneously, assigning a dynamic threat score to each actor based on behavior across all vectors. The system is modeled on the MITRE ATT&CK framework and includes a predictive blocking feature that can sever connections based on attack trajectory before any single threshold is breached. CNSL targets both Linux servers and Kubernetes environments, aiming to stop coordinated, multi-stage intrusions that traditional intrusion prevention systems typically miss.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Five Prompts to Make AI-Generated Bug Fixes Easier to Verify

Developers using OpenAI's Codex for bug fixes often receive plausible-sounding explanations that may not fully resolve the underlying issue. A set of five adaptable prompts has been outlined to help engineers verify Codex-generated fixes more rigorously within existing repositories. The prompts cover targeted fix scoping, file-level inspection, regression test creation, diff review, and structured handoff summaries. Each prompt is designed to make both the fix and its verification transparent, reducing the risk of incomplete or misleading results. The guidance uses an illustrative empty-cart checkout scenario to demonstrate how vague instructions can be sharpened into verifiable, bounded tasks.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developers Share Real Costs of Silent Software Failures That Go Undetected

A discussion on DEV Community is asking software developers to share concrete data on the real-world impact of silent failures — bugs or processes that report success while actually doing nothing. The thread seeks specific figures such as detection time, financial losses, client churn, or hours spent diagnosing the issue. The conversation was prompted by an observation that shortening the diagnosis time for such failures could reduce consulting invoices, potentially making a dedicated tool commercially viable. Contributors are encouraged to share outcomes ranging from costly incidents to near-misses caught quickly. The goal is to gather hard numbers rather than general patterns to inform whether building a detection tool would be worthwhile.

0 comments Read more at DEV Community