SShortSingh.
Back to feed

Developer runs 744B GLM 5.2 model on a 32GB RAM laptop using disk-streaming trick

0
·1 views

A solo developer has built an open-source inference engine called Colibrì that allows the massive 744-billion-parameter GLM 5.2 language model to run on a standard laptop with around 32GB of RAM. The project works by keeping only the dense components of the model resident in RAM at int4 precision, while streaming the model's 21,504 routed expert layers on demand from disk storage. The entire engine is written in a single C file of roughly 1,300 lines, requiring no GPU, no Python runtime, and no external BLAS libraries. Although inference is slow — around 0.1 tokens per second on a 12-core laptop — the developer's stated goal was simply to make the model run at all on consumer hardware. The project is available on GitHub and the developer is welcoming feedback and potential contributors.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Claude Code vs. OpenAI Codex: How the Two AI Coding Agents Compare

Claude Code, built by Anthropic, and Codex, developed by OpenAI, are two of the most advanced AI coding assistants available today, each capable of understanding entire codebases, writing tests, fixing bugs, and executing multi-file edits. Both tools support a wide range of programming languages and offer strong performance in code generation, repository awareness, and agentic workflows. Claude Code is particularly noted for its detailed reasoning, documentation generation, and ability to help developers navigate complex architectural challenges through natural language interaction. Codex, by contrast, is oriented toward efficient task execution, making it well-suited for implementing features, resolving issues, and maintaining production code with minimal prompting. The choice between the two largely depends on whether a developer prioritizes deep collaborative reasoning or streamlined, autonomous engineering task completion.

0
ProgrammingDEV Community ·

Rosemary Tool Tunnels Network Traffic Over QUIC With Kernel-Level Transparency

Rosemary is an open-source networking tool that intercepts traffic at the kernel level and tunnels it over encrypted QUIC connections through a lightweight remote agent. It supports TCP, UDP, ICMP, and DNS without requiring proxy settings, TUN devices, or tools like proxychains. Users can route all traffic through a chosen remote host, enabling their local machine to behave as if it were on the remote network. The tool works across Linux, Windows, macOS, FreeBSD, and OpenBSD, and agents do not require root privileges. A web dashboard, REST API, and multi-hop pivoting support are also included, with traffic secured using AES-256-GCM encryption.

0
ProgrammingDEV Community ·

Developer shares three methods to integrate external payments into Base44 apps

Oded, co-founder of payment platform UniPaaS, has outlined three documented ways to add external payment providers to apps built on the no-code platform Base44. The guide addresses a common builder frustration: built-in options like Stripe and Base44 Payments require verified business accounts in supported countries, leaving many developers unable to accept live payments. The most-requested feature on Base44's feedback board is support for additional payment providers beyond Stripe. The three integration methods covered are MCP connections for account setup during building, OpenAPI-based runtime integrations managed by a workspace admin, and backend functions for full developer control at runtime. The author discloses a conflict of interest, noting the examples use his own product, but states the Base44 integration surfaces work with any external payments API.

Developer runs 744B GLM 5.2 model on a 32GB RAM laptop using disk-streaming trick · ShortSingh