Engineer ports Gemma-4 26B MoE model to AWS Inferentia2, solves silent rank-zero bug

A developer porting Google's Gemma-4 26B-A4B mixture-of-experts model to AWS Inferentia2 hardware encountered a silent failure where the model compiled successfully but produced empty output instead of text. The root cause was that AWS Neuron's ModelBuilder compiles only rank zero and replicates it, meaning all eight tensor-parallel ranks received the same wrong slice of expert weights rather than their unique shards. The fix involved replacing a dynamic per-rank weight scatter with a static, pre-sharded dense expert matrix that could be correctly traced into Neuron's static HLO graph. The engineer also resolved the model's sparse top-8 routing loop — which cannot be traced statically — by computing all 128 experts densely and zeroing out non-selected ones, making the math equivalent while remaining hardware-compatible. The final model runs correctly on an inf2.24xlarge instance, matching CPU fp32 reference output with a prefill latency of 77 milliseconds.