SShortSingh.
Back to feed

Five AI models scored 98% or higher on a practice certification exam

0
·5 views

The developer behind ClaudeArena, a free practice site for Anthropic's Claude Certified Architect exams, tested whether AI agents could pass the Professional exam by giving them a browser and having them answer from memory. Five models from four makers, including GPT 5.6 Luna and Claude Fable 5.2, scored 98% or 100% on the 63-question Hard exam, with some finishing in under six minutes. Results varied: one model dropped to 84% on a repeat run, and self-reports from the agents often did not match the server's records. Both Qwen and DeepSeek missed the same question, which the developer verified against Anthropic's documentation. The experiment also revealed an accessibility flaw, as agents could not reliably tell which answers were selected, and the site has since announced chosen answers to fix it.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Guide Shows How to Write a Game Prompt That Gets the Game You Imagine

The article explains that a strong prompt for an AI game engine should specify five elements: the core player action, the pressure or obstacle, the way to win, a concrete visual style, and one defining twist. It argues that vague requests produce generic results, while precise ones give the engine a clear direction, and that a single well-built sentence is enough to start. Weak and strong examples are provided for each element to show the difference. The guide also warns against feature lists, unexplained references to other games, and mood-only descriptions that lack a mechanic. It notes that the game can be refined afterward through conversation with the engine.

0
ProgrammingDEV Community ·

Preventing Duplicate Weekly Digest Webhooks After Queue Lease Timeouts in Node.js

A design guide argues that delayed weekly webhook deliveries should separate scheduling from delivery, using a durable delivery record and a lease shorter than the queue timeout. It recommends a single idempotency key across all retries, since a timeout does not confirm whether the receiver got the request. The author uses a hypothetical 100,000-customer workload with an assumed 2% retry rate to show that duplicate attempts drive costs more than the schedule itself. The guide advises storing only compact delivery outcomes rather than full payloads, accepting that regenerating payloads during investigations will be slower. It also stresses renewing leases with a cap and keeping the HTTP deadline inside the remaining lease window.

0
ProgrammingHacker News ·

Archive Team to preserve Apple Support Community before its November 2026 closure

The Archive Team, a volunteer digital preservation group, has launched an effort to archive the Apple Support Community. The forum is scheduled to shut down on November 20, 2026. The project aims to save the discussions and user-generated content hosted on the platform. Details are documented on the Archive Team wiki. The item gained modest attention on Hacker News, with a handful of points and comments.

0
ProgrammingHacker News ·

Microsoft's ONNX Runtime repository appears to be missing from GitHub

A Hacker News post reports that Microsoft's ONNX Runtime project is no longer visible on GitHub. The post links to the project's repository page and to a discussion thread that has gathered 24 points and 3 comments. The source material offers no explanation for the apparent removal and does not say whether it is temporary or permanent. ONNX Runtime is a widely used open-source engine for running machine learning models, so the report draws attention to its availability. Further details have not been confirmed in the available information.