MiMo v2.5 Hybrid SWA Cuts Model Size 60% While Retaining 98% Accuracy

MiMo v2.5 introduces a hybrid inference optimization technique that combines Stochastic Weight Averaging with dynamic pruning and quantization. The approach achieves a 60% reduction in model size and a 70% boost in inference speed while retaining 98% of the original model's accuracy. Three core innovations drive these gains: gradient-guided adaptive weight averaging, second-order gradient-based latency-aware pruning, and mixed-precision quantization using 8-bit integers for dense layers and 16-bit for recurrent components. The architecture is particularly suited for edge devices, real-time inference pipelines, and models that require frequent retraining. Developers are advised against using it where full-precision outputs or latency-insensitive batch processing are priorities.