OpenAI Drops Recommendation for SWE-Bench Pro Coding Evaluation

·2 views

OpenAI has publicly withdrawn its recommendation of SWE-Bench Pro as a reliable coding benchmark. The company addressed concerns about distinguishing genuine model performance from noise in coding evaluations. OpenAI published a blog post outlining its updated stance on how coding benchmarks should be assessed. The move signals a broader effort by OpenAI to improve transparency and rigor around AI performance measurement in software engineering tasks.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

OpenBSD flaw, GitHub AI data leak, and CDN supply chain risk top security news

A critical use-after-free vulnerability (CVE-2026-57589) in OpenBSD allows local attackers to escalate privileges to root, prompting urgent patch advisories for system administrators. Separately, security researchers demonstrated a prompt injection attack called 'GitLost' that manipulated GitHub's AI agent into leaking code from private repositories. The exploit bypassed safeguards by crafting malicious inputs that redirected the AI agent's behavior, exposing confidential data. A third incident involved an obfuscated, self-evaluating bash script discovered embedded in data linked to a Uniqlo t-shirt and served via Akamai's CDN, raising supply chain integrity concerns. Together, the three cases highlight growing security risks across operating systems, AI integrations, and content delivery infrastructure.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

IsItCrashing.com Launches Tool to Detect Broken Pages Before Users Do

A new web tool called IsItCrashing.com has been launched to help developers identify site errors before they affect end users. The platform scans websites for common HTTP errors such as 404 and 500 status codes. It generates a clean, readable report to help developers quickly locate and fix broken pages. The tool aims to give developers greater confidence when deploying websites by catching issues proactively.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Structured AI Coding Workflow Using Staged Pipelines Over Chat Prompts

A software developer has shared how they replaced ad-hoc AI-assisted coding with a structured, stage-based workflow using an open-source tool called agentic-skills. The approach moves tasks through defined phases — requirements, architecture, implementation, checks, review, and pull request creation — with each stage producing inspectable artifacts. Unlike typical prompt-and-response AI coding sessions, this pipeline allows work to be paused and resumed without losing context. The setup separates shared orchestration logic from project-specific configuration, keeping individual repositories lightweight while centralizing pipeline rules. The developer argues this shift moves the focus from crafting prompts to evaluating concrete outputs at each stage of the engineering process.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

XModelBuilder library eliminates need for hand-written test data builders in .NET

A .NET library called XModelBuilder offers a generic fluent builder that works with any C# class, removing the need to write and maintain separate builder classes per type. Developers can set only the properties relevant to their tests while the library auto-fills the rest, including deeply nested paths and collection indexers. The library supports reproducible random data through a seeded faker integration, preventing unpredictable test failures caused by changing values across runs. A BuildMany method allows batch creation of test objects with shared or index-varying properties in a single call. Optional per-type builder classes can be registered to define defaults, and multiple named presets for the same type can be configured and switched as the default.

0 comments Read more at DEV Community