176 KB C Binary Runs 2.78-Trillion-Parameter AI Model on 8 GB RAM Without a GPU

Developer Fareed Khan has built kimi-k3-in-c, a 176 KB pure C99 binary that runs Moonshot AI's Kimi K3 model — which has 2.78 trillion parameters and normally requires dozens of H100 GPUs — on a single CPU with just 8 GB of RAM. The project achieves this through four layered optimisations: 4-bit weight packing, on-demand expert loading from NVMe storage, dense-layer streaming with O_DIRECT, and an LRU cache for routed experts, collectively reducing memory requirements by 676 times. Crucially, the engine produces output byte-for-byte identical to the PyTorch reference, meaning no accuracy is sacrificed despite the extreme resource constraints. Performance is slow — around 33 seconds per token at the minimum memory preset — making it impractical for real-time use, and generating 200 tokens can take roughly two hours. Khan has described the project explicitly as a teaching artifact rather than a production inference server, demonstrating the structural sparsity properties of Mixture-of-Experts models.