7 Local LLMs Benchmarked on NVIDIA DGX Spark: Bigger Models Don't Always Win

A Microsoft MVP based in Japan tested seven major local large language models by running the exact same Japanese business question through each one on a single NVIDIA DGX Spark machine. The benchmark evaluated not just generation speed in tokens per second, but also format compliance, factual reliability, and appropriate caveats for real business use. Results showed no clear correlation between model size and output quality — one fast model produced answers too risky for customer-facing use, while a 120B-parameter model returned a completely empty response field. NVIDIA's Nemotron 3 Super 120B-A12B, requiring 87GB of weights, was directly compared against smaller models in the 23GB class to assess whether its size justified its resource cost. All tests were conducted under identical fixed conditions — temperature zero, the same seed, and a capped generation budget — to isolate model differences from configuration variables.