Moonshot AI Launches Kimi K3: 2.8T Parameter Open MoE Model With 1M Token Context

Moonshot AI has released Kimi K3, an open Mixture-of-Experts large language model with 2.8 trillion parameters and support for a one-million-token context window. The model introduces Kimi Delta Attention, a hybrid linear attention mechanism that enables up to 6.3 times faster decoding on long-context tasks. A second architectural innovation called Attention Residuals selectively reuses representations across network layers, reportedly boosting training efficiency by around 25 percent. Kimi K3 also features Stable LatentMoE with Quantile Balancing, a new expert-routing approach that achieves approximately 2.5 times better scaling efficiency compared to its predecessor, Kimi K2. In published benchmarks, the model outperforms competing systems in 6 of 35 categories, though it still trails in certain areas such as FrontierSWE and HLE-Full.