Running a 2 GB Model on an 8 GB Edge Device Depends on More Than File Size

A 2 GB model file on an 8 GB device does not leave 6 GB of usable inference memory, because weights, KV cache, compute buffers, and the operating system all compete for RAM. The article argues that a single short chat says little about capacity, and that the longest request and peak concurrency must be tested. Storage size and theoretical KV-cache calculations can be computed beforehand, but the real device peak, runtime overhead, and recovery behavior require measurement on the target hardware. It also notes that weight quantization does not reduce other memory consumers by the same factor, and that terms like 'needs 4 GB' can refer to different quantities. The example is hypothetical and not a tested board result.