Developer Runs Three Gemma 4 AI Runtimes on One AWS GPU for Under $3

A developer benchmarked three different inference runtimes — vLLM, JAX, and PyTorch with Transformers — each serving Google's Gemma 4 2B model on a single AWS g5g.2xlarge instance equipped with an NVIDIA T4G GPU. The experiment used a shared Python benchmark harness across all three deployments to ensure the runtime was the only variable, correcting an earlier flaw where each rig measured itself independently. The entire exercise, spanning 19 instances and roughly 4.5 instance-hours, cost less than three dollars in AWS spot compute. The g5g instance family is notable as the only AWS offering that pairs an NVIDIA GPU with an ARM-based Graviton2 host, making it a unique testbed for aarch64 inference. The project also surfaced five incorrect performance claims that only measurement — not assumption — could catch, highlighting the importance of a unified benchmarking approach.