New Benchmark Tests AI Coding Agents on Real 2026 Pull Requests Across 5 Languages

·3 views

A team built 'octobench', a coding benchmark using 25 real tasks drawn from pull requests merged in 2026 across Python, PHP, Rust, C++, and JavaScript open-source projects, to avoid the contamination and arbitrary grading problems found in existing benchmarks. Each task reconstructs the repository state just before a fix was merged, with agents working like contractors and being evaluated by the project's own held-out tests. Octomind's agent using an open model scored 24 out of 25, outperforming Claude Code with Opus, while the same underlying model in a different setup solved only 19 tasks at twice the cost. The benchmark was designed to stay fresh by targeting recently merged fixes, several of which were harvested within days of being committed. The results suggest that agent architecture and tooling matter as much as the underlying model when tackling real-world software engineering tasks.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Playground API v2 Adds Delay Simulation, Error Injection, and Postman Export

Playground API, a zero-config stateful mock REST API for frontend developers, has received a major v2.0 update days after its initial launch. The update introduces network delay simulation via query parameters or request headers, allowing developers to test loading states and spinners more realistically. Developers can now also inject arbitrary HTTP error responses on demand to verify how their applications handle failures like 500 or 404 errors. Version 2 adds nested sub-resource routing, full-text search, dynamic sorting, and header-based session identity for use with E2E test runners like Playwright and Cypress. The release also includes downloadable API specs for Postman, Bruno, and Insomnia, along with auto-generated code snippets in nine programming languages.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Browser Agents: The Real AI Battle Is Over Your Sessions, Clicks, and Intent

Major AI labs including OpenAI, Google, and Anthropic are competing not to own a browser brand but to control the agent layer that sits atop users' active sessions, open tabs, and purchase decisions. Unlike standard chatbots that only know what a user types, browser agents can observe real-time activity such as half-filled carts, open dashboards, and abandoned forms. OpenAI's ChatGPT Atlas illustrates the shifting strategy — launched as a standalone browser, it was later folded into the desktop app and extensions, suggesting the durable prize is the embedded agent, not the browser shell. Google continues deepening Gemini into Chrome, while Anthropic ships browser-operation capabilities without owning a browser at all. Analysts warn that controlling default browser positions, extensions, or session context is a significant commercial moat, since the browser is where purchase intent and search revenue are generated.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

React Mastery Day 12: How Controlled Components Power Forms in React

Day 12 of the React Mastery Series focuses on building forms using React's controlled component pattern. Unlike traditional HTML forms where the browser manages input values, React stores form data in component state, making it the single source of truth. Developers use the value and onChange properties together to keep inputs in sync with state, and a single state object can manage multiple fields using JavaScript's computed property name syntax. Form submissions are handled by calling event.preventDefault() to stop the default browser page refresh, allowing developers to validate data and call APIs instead. The article also covers basic client-side validation techniques such as checking required fields, email format, and password rules using conditional rendering.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

WooCommerce.com Cuts Load Times 40% by Selectively Loading Plugins Per Request

WooCommerce Engineering published findings showing that loading only request-relevant plugins significantly reduces WordPress bootstrap costs on WooCommerce.com. The team found that unrelated plugins consumed 10–20% of uncached page generation time, and selective loading cut memory use by over 50% on certain endpoints. Request times fell from roughly 800ms to 475ms and 880ms to 550ms on targeted routes, with product pages seeing around a 10% improvement. The approach uses an early MU-plugin that filters WordPress's active plugin list before regular plugins are included, meaning excluded plugins never execute. Several existing WordPress tools such as Plugin Organizer and Freesoul Deactivate Plugins offer similar functionality, though their underlying architectures differ in how and where the selection logic is prepared.

0 comments Read more at DEV Community