Edge0 runs 35B AI model in under 3 GB RAM by streaming experts from SSD

Edge0-AI has open-sourced Edge0, a streaming inference engine licensed under Apache 2.0 that runs large language models with minimal RAM by keeping model weights on an SSD and loading only the active expert parameters per token. The system leverages sparse Mixture-of-Experts (MoE) architecture, where just 3 billion of a model's 35 billion parameters are active at any given moment, capping peak memory use at around 2.9 GB. Two preview models are available: Edge0-35B-A3B, based on Qwen3.5-MoE, and Edge0-8B-A1B, built on inclusionAI's Ling 3.0, both shipped as 4-bit quantized checkpoints. Benchmarks recorded on a Mac Mini M4 Pro show decode speeds of roughly 15–18 tokens per second for the 35B model, though memory usage rises with longer prompts as the KV cache grows. The engine currently supports Apple Silicon Macs running Python 3.10 or later and is installable via a single pip command from the project's GitHub repository.