Local AI on Consumer GPUs: What Actually Works and What Doesn't in 2025

A hands-on evaluation of running AI models locally on consumer hardware, drawing on six independent benchmarking sources, finds that available RAM and VRAM are the most critical specs to consider. Quantization — compressing model weights from 16/32-bit down to 4/8-bit — makes large models feasible on ordinary machines, reducing a 70B model's memory requirement from 140 GB to as little as 30 GB. Code autocomplete emerged as the strongest local use case, with models like Qwen 2.5 Coder 7B delivering sub-100ms responses even on low-VRAM GPUs, while video generation was rated slow and disappointing even on high-end hardware. On hardware choice, unified-memory systems such as Apple M-series or AMD Strix Halo offer more capacity per dollar, whereas dedicated GPUs like the RTX 4090 deliver two to three times faster inference at the cost of lower total memory. The practical advice for most users: check available RAM first, run the largest model in the 7B–14B range that fits, and expect the model landscape to improve significantly every few months.