Colibri Runs 744B GLM-5.2 AI Model on 25GB RAM Using Single-File Pure C Engine

·1 views

A developer named Nokka published Colibri, a single-file inference engine written in roughly 2,400 lines of pure C with zero external dependencies, capable of running GLM-5.2, a 744-billion-parameter Mixture-of-Experts model, on a consumer machine with just 25GB of RAM. The engine works by keeping only the dense portion of the model (about 9.9GB) permanently in RAM while streaming the remaining 21,504 routed experts from NVMe storage on demand as each token is processed. GLM-5.2, released on June 13, 2026 by Z.ai under an MIT license, activates only around 40 billion parameters per token despite its massive total size, which makes the disk-streaming approach viable. The system loads and becomes ready to respond in approximately 32 seconds, generating text at 0.05–0.1 tokens per second on a 12-core WSL2 development machine. Colibri also implements compressed KV-cache persistence to disk, allowing conversation history to survive restarts without reloading the full model context.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Go Middleware Ordering Rules That Prevent Silent Production Failures

A production API ran smoothly for six months before a misplaced panic-recovery middleware caused 2% of requests to silently return 500 errors with no logs or traces. The root cause was a recover middleware placed inside the logging layer, causing it to swallow panics after the logger had written its line but before a request ID was set. Based on 14 years of running Go APIs, the author outlines three non-negotiable middleware ordering rules: Recover must be the outermost layer, Request ID must precede the logger, and Auth or rate-limiting should sit closest to the handler but after observability layers. A small chaining helper is recommended to avoid deeply nested, hard-to-read middleware declarations. Getting the order wrong inverts the entire execution flow, turning trivial ten-line functions into hard-to-debug production hazards.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Adds SigNoz Observability to MERN Stack App Using OpenTelemetry

A developer integrated SigNoz, an open-source observability platform, into RoaVista, a full-stack room booking application built on the MERN stack. The setup involved self-hosting SigNoz via Docker and instrumenting the Node.js/Express backend using OpenTelemetry to push trace data to the SigNoz OTLP endpoint. The standout feature was distributed tracing, which provided visual flame graphs breaking down request latency across MongoDB queries, Express middleware, and authentication steps. Previously, a slow API route offered no clear indication of where the bottleneck occurred, but tracing made it possible to pinpoint delays and errors at each stage. The experience shifted the developer's approach from reactive debugging with console logs to proactive, metrics-driven monitoring.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Python Class Variable Mutation Trap Explained for Experienced Developers

A common Python pitfall involves mutable class variables being shared across all instances, producing unexpected behavior. When a list is defined as a class attribute, all instances reference the same object, so mutations like append affect every instance simultaneously. This differs from immutable types such as integers, where augmented assignment creates a new instance variable rather than modifying the class-level object. The correct fix is to initialize mutable attributes inside __init__, ensuring each instance gets its own copy. Class variables remain appropriate for constants, shared configuration, or carefully managed class-wide counters when modified explicitly through the class name.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Web Bot Auth lets AI agents prove their identity cryptographically on every request

A new authentication mechanism called Web Bot Auth, developed through the IETF and already deployed by Cloudflare, allows AI agents to cryptographically sign each web request using a private key, with the corresponding public key published at a standardized URL. Receiving servers or Cloudflare's edge infrastructure can then verify the signature, confirming the agent's identity without relying on easily spoofed user-agent strings or unstable IP ranges. Cloudflare launched its signed agent program in August 2025, initially registering ChatGPT agent, Goose, Browserbase, and Anchor Browser in a publicly accessible directory on Cloudflare Radar. Anthropic's Claude is not yet registered in the directory, meaning sites using Cloudflare's Block AI Bots rule can currently block Claude entirely, forcing operators to add manual exceptions. Once Claude receives a verified entry, websites will be able to grant or restrict agent access based on confirmed identity rather than unverifiable claims.

0 comments Read more at DEV Community