Developer runs 744B GLM 5.2 model on a 32GB RAM laptop using disk-streaming trick
A solo developer has built an open-source inference engine called Colibrì that allows the massive 744-billion-parameter GLM 5.2 language model to run on a standard laptop with around 32GB of RAM. The project works by keeping only the dense components of the model resident in RAM at int4 precision, while streaming the model's 21,504 routed expert layers on demand from disk storage. The entire engine is written in a single C file of roughly 1,300 lines, requiring no GPU, no Python runtime, and no external BLAS libraries. Although inference is slow — around 0.1 tokens per second on a 12-core laptop — the developer's stated goal was simply to make the model run at all on consumer hardware. The project is available on GitHub and the developer is welcoming feedback and potential contributors.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in