Microsoft Open-Sources BitNet Framework to Run 100B-Parameter LLMs on a Single CPU

Microsoft Research has open-sourced BitNet, an inference framework for 1-bit large language models, under the MIT License on GitHub. The framework uses 1.58-bit quantization, storing model weights as ternary values (-1, 0, +1), which dramatically reduces memory requirements compared to standard 16-bit or 32-bit models. According to Microsoft's README, BitNet can run a 100-billion-parameter model on a single CPU at speeds of 5–7 tokens per second, while delivering 2.37x–6.17x speedups over llama.cpp on x86 processors and reducing energy consumption by up to 82%. The flagship released model, BitNet-b1.58-2B-4T, has 2.4 billion parameters trained on 4 trillion tokens, though the 100B capability remains a projection from a technical report rather than a publicly available model. Notably, BitNet requires models to be trained from scratch using its architecture — existing models like Llama or DeepSeek cannot simply be re-quantized to 1.58-bit without significant quality loss.