GPU Has Two Parts: Why VRAM, Not Cores, Determines If Your Model Loads

A GPU card consists of two distinct components: processing cores that perform calculations, and VRAM (video memory) that stores data — a distinction that confuses many engineers new to AI infrastructure. When a large language model is deployed, its weights are loaded into VRAM, not into the cores themselves. The cores continuously stream data from VRAM to perform each calculation, which is why language models generate text word by word rather than all at once. On AWS EC2 instances, the prominently listed system RAM figure is irrelevant to model loading — only the GPU's VRAM capacity determines whether a model fits. For example, a g6.4xlarge instance lists 64 GiB of system RAM but only 24 GB of VRAM on its NVIDIA L4 GPU, and AWS does not display VRAM directly on its EC2 pricing page, requiring a separate cross-reference.