Five AI Models Tested Locally on Coding Task; Two Failed Before Even Starting
A developer running a local AI model benchmark series tested five language models — Qwen 3.6 27B, Qwen 3.6 35B-A3B, Qwythos-9B, GLM-4.7-Flash, and Nemotron-3-Nano — on a real-world coding task using an RTX 5090 GPU with no cloud services. The task required building a Tag Manager feature for an admin panel, covering API endpoints, a frontend page, Playwright screenshots, and a clean build before committing. The field expanded from a planned three models to five, as adding extra contestants cost little once the testing pipeline was already running. Two models, Qwythos-9B and Nemotron-3-Nano, crashed with hard errors on their very first request due to bugs in llama.cpp's tool-call parser, requiring the tester to patch templates mid-benchmark. The round was also a rematch for Qwen 3.6 35B-A3B, which had previously built an entire feature in Round 7 but failed at the final screenshot step, leaving all its work uncommitted.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in