Open-weight models trail hosted rivals as coding agents in new benchmark runs

A reviewer ran open-weight models as coding agents on two agentic benchmarks and found none matched hosted systems this month. Meta's 816B-parameter Muse Spark Open, released 15 October, scored 0.091 and 0.0877 on the Forge 2.0 build, far below the board's top score of 0.8523, and produced no verdict on the Gauntlet 7.2 test. Locally run models including Google's Gemma 4 and IBM's Granite 4.2 did no better, and several pilot runs were cut short or fell below the reviewer's bar. The reviewer noted that recurring failures, such as large files never being written and tool calls returning as prose, were often tied to the test setup as well as the models. Meta's own vendor claims for the model were not matched in these independent runs.