Five AI models scored 98% or higher on a practice certification exam
The developer behind ClaudeArena, a free practice site for Anthropic's Claude Certified Architect exams, tested whether AI agents could pass the Professional exam by giving them a browser and having them answer from memory. Five models from four makers, including GPT 5.6 Luna and Claude Fable 5.2, scored 98% or 100% on the 63-question Hard exam, with some finishing in under six minutes. Results varied: one model dropped to 84% on a repeat run, and self-reports from the agents often did not match the server's records. Both Qwen and DeepSeek missed the same question, which the developer verified against Anthropic's documentation. The experiment also revealed an accessibility flaw, as agents could not reliably tell which answers were selected, and the site has since announced chosen answers to fix it.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in