Alibaba Cloud Launches Qwen2 Open-Source LLM Series with 128K Context Support

Alibaba Cloud has released Qwen2, a new family of open-source large language models ranging from 0.5 billion to 72 billion parameters, including a Mixture-of-Experts variant. The models were trained on a dataset covering 27 languages in addition to English and Chinese, making them suitable for multilingual and global applications. A standout feature is support for context windows of up to 128K tokens, enabled partly by Group Query Attention architecture, which benefits tasks like document analysis and multi-turn conversations. Qwen2 models demonstrate strong benchmark results in language understanding, coding, and mathematics, with the larger models achieving state-of-the-art performance. The series is publicly available on Hugging Face and integrates with the standard Transformers library, allowing developers to deploy the models locally or on their own infrastructure.