GPT-6 Astra Scores 99.9% on ARC-AGI-3, But Harness Choice Explains the Gap

·1 views

On 3 September 2026, ARC Prize published official ARC-AGI-3 benchmark results for GPT-6 Astra, showing two distinct scores depending on which evaluation harness was used. Under OpenAI's Provider Adapter at high reasoning, Astra scored 99.9% at a cost of $18,817, while the shared Standard harness at maximum reasoning yielded only 62.7% at a higher cost of $26,098. The key difference lies in memory handling: the Provider Adapter preserves reasoning state and compresses context between requests, allowing the model to reuse prior work, whereas the Standard harness does not. ARC Prize explicitly declined to interpret the result as proof of AGI, with co-founder Mike Knoop stating there is insufficient evidence to make that claim. Both scores remain on the leaderboard with harness labels, and ARC Prize has cautioned that any single unlabeled percentage from this benchmark is not a valid basis for comparison.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

World Models Move Beyond Robotics Into Real-Time AI Video Generation

World models, a concept in AI research describing systems that learn how environments change over time, are gaining renewed attention beyond their traditional use in robotics and self-driving vehicles. Unlike large language models, which predict sequences of text tokens, world models learn causal and physical patterns by processing video and simulated interactions. AI researcher Yann LeCun has described this approach as analogous to the internal mental maps humans build through repeated experience and cause-and-effect reasoning. Video generation company Runway has now applied this technology in its GWM Worlds 2 model, which allows users to steer and shape a video environment in real time rather than simply prompting a finished output. The development signals a broader shift in generative AI toward systems capable of understanding and simulating physical reality, not just language.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How SaaS Teams Should Own OTP Templates for Secure Password Resets

For B2B SaaS password-reset flows, security experts recommend keeping OTP template logic and challenge state within the application itself rather than delegating it to third-party delivery vendors. Both SMS and email are viable channels, but the choice should be treated as a delivery adapter decision, not a security one — the application must control code lifetime, content, and rate limits. A single, atomically consumed challenge record is advised to prevent race conditions where multiple delivery channels could each hold a valid code simultaneously. Template ownership is framed as a core security boundary, since the message content itself can be a vector for user confusion or attacker exploitation. Retention policies for challenge logs should be made configurable to accommodate differing US and EU legal requirements, and reset flows must require a previously verified destination address.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Free macOS App to Track and Identify AI Coding Agent Processes

A developer frustrated by the inability to distinguish AI coding agent processes in macOS Activity Monitor has built Corral, a free open-source native app. Corral identifies and displays processes from tools like Claude Code, Cursor, Codex, Kiro, and Antigravity, showing which project each belongs to, how long it has been running, and its resource usage. The app is built with Swift and SwiftUI, carries an MIT license, requires no special permissions, and makes no network calls. Because process name matching proved unreliable — agents often run under generic names like 'node' — the developer used install directory paths as the stable identifier instead. Each supported tool stores session state differently, requiring custom parsing logic for log files, rollout files, and SQLite databases to surface meaningful session information.

0 comments Read more at DEV Community

ProgrammingGitHub Blog ·

GitHub Copilot App Lets Beginners Review Code, Terminal, and Browser Side by Side

GitHub has published a beginner-focused guide on using the GitHub Copilot app more efficiently. The tutorial addresses a common workflow challenge: reviewing AI-generated code typically requires switching between multiple tabs or windows. The guide walks users through how to view code diffs, execute terminal commands, and preview web applications within a single interface. The side-by-side layout is designed to streamline the process of checking and validating agent-generated code.

0 comments Read more at GitHub Blog