ReBA scheduler fixes expert load imbalance in vision-language AI models

Researchers have proposed ReBA, a geometry-guided routing scheduler designed to address load imbalance in vision-language mixture-of-experts (MoE) models. The core problem is that existing routing methods treat all tokens equally, allowing high-resolution image patches to overwhelm certain experts while text tokens remain underutilized. ReBA solves this by enforcing separate routing pathways for visual and linguistic tokens and applying equal weighting per image instance. In tests across four model backbones, ReBA reduced layer-level load variation more consistently than both the standard auxiliary loss method and a coupled image-instance baseline. The study notes limitations, including the absence of wall-clock speed or memory benchmarks and untested performance on very large-scale deployments.