NVIDIA Research Argues Small Language Models Suit Most AI Agent Tasks

NVIDIA Research published a position paper arguing that models under 10 billion parameters are sufficient, better suited, and far cheaper for most calls made by AI agents. The paper claims serving a 7-billion-parameter model can be 10 to 30 times cheaper than a 70-to-175-billion-parameter model, and that specializing a small model takes only a few GPU-hours. An independent 2026 DFKI study found the largest model tested used about 32 times the energy of a small one for roughly a 2-point accuracy gain on a math benchmark. The same study showed smaller models sometimes consumed more energy overall because they produced longer outputs in a substantial share of comparisons. Analyst data cited in the article indicate infrastructure limits are a major barrier to AI deployment for many organizations.