Study compares ten weight formats for Gemma 4 E2B on AMD MI300X hardware

A developer has conducted a performance comparison of ten different weight formats for serving the Gemma 4 E2B language model on an AMD Instinct MI300X GPU. The test used the vLLM serving framework, timing each format across various request counts and prompt lengths. Results showed FP8 format performed closest to the baseline BF16 format, while 4-bit formats were significantly slower. The fastest formats involve the most rounding of the model's original trained weights, presenting a trade-off between speed and precision. All test logs, reports, and scripts have been publicly released on GitHub.