Engineer runs Google Gemma 4 on AWS G5g with Graviton2 and NVIDIA T4G GPU

A developer published a technical field report detailing the process of deploying Google's Gemma 4 E2B model on an AWS EC2 G5g instance, which pairs an ARM-based Graviton2 CPU with an NVIDIA T4G GPU. The G5g is the only AWS instance to combine an NVIDIA GPU with a Graviton processor, making it an unusual and poorly documented target for modern AI workloads. Three core obstacles emerged: the T4G's compute capability 7.5 (SM 7.5) is absent from arm64 builds of mainstream inference stacks like vLLM, a 64 KiB shared memory ceiling halts model execution, and no released vLLM version pins the required PyTorch 2.12. After building vLLM from source and applying a patch, the setup achieved 43.1 tokens per second in single-stream greedy decoding with a 329,579-token KV cache, with AWS's own Deep Learning ARM64 AMI proving critical as it retains SM 7.5 support through PyTorch 2.12 on CUDA 13.2.