How to Choose a Consumer GPU for Running Local LLMs by VRAM Tier

Selecting a GPU for running large language models locally comes down to two key factors: how much VRAM a card has and its memory bandwidth. A practical formula can estimate the maximum model size a given card can hold, factoring in KV cache, activations, overhead, and quantization precision. Running a model entirely within VRAM — even at lower precision — is almost always faster than allowing any portion to spill into system memory via PCIe. Quantization can degrade model quality in task-specific ways, so users are advised to test on their own prompts rather than assuming a tier is sufficient. Buying local hardware is better suited to continuous, high-volume use rather than occasional inference, making it more of a privacy and control decision than a straightforward cost saving.