MoE AI models achieve efficiency with sparse activation, dominating open-weight frontier

Recent open-weight AI models from DeepSeek and Xiaomi utilize Mixture-of-Experts architectures for significant efficiency gains. These models, released in September 2026, activate only a tiny fraction of their total parameters per token, reducing computational cost. The technique, pioneered in 2017, routes each input token to a small subset of specialized 'expert' networks rather than engaging the entire model. This approach allows models to have vast knowledge capacity while maintaining lower operational costs, making large models more practical to run.