Developer Benchmarks Local LLMs on Consumer Hardware, Picks Gemma 4 26B as Top Model

A developer has documented running large language models locally on two consumer-grade desktop machines, avoiding cloud APIs for most tasks. The primary setup uses a Ryzen 5950X with an AMD RX 6900XT GPU running Gemma 4 26B, a mixture-of-experts model quantized to Q4_K_M and split across GPU and CPU due to VRAM constraints. Several models were benchmarked for both throughput and output quality using a custom five- and ten-task evaluation suite. Gemma 4 26B achieved a near-perfect score of 99/100 on the harder test while sustaining around 17 tokens per second, outperforming faster but shallower smaller models. The project concludes that speed and quality involve a clear trade-off, with future posts planned to cover context length, KV-cache optimisation, and model residency strategies.