A 65B Local Model Outscored a 501B Model on Identical Eval Questions

A developer reported that a 65-billion-parameter model run locally scored 80.52 on their evaluation suite, higher than a 501-billion-parameter frontier model on the same test. Both models answered the same 200 multiple-choice questions at temperature 0 with no system-prompt changes, using the same scoring script. The local model was quantized to Q4_K_M and ran on a single consumer GPU, while the larger model was accessed through its hosted API with default settings. The smaller model reportedly won 14 of 20 categories. The author noted that verifying the larger model's result costs about twelve dollars per full run, compared with roughly forty minutes of GPU time and electricity for the local run, and shared a short video walkthrough of the setup.