LLMs Now Run Locally on $8 Microcontrollers at Near-Reading Speed

·1 views

On July 25, 2026, a developer known as slvDev published esp32-ai, demonstrating a 28.9-million-parameter language model running entirely on an ESP32-S3 microcontroller at 9.88 tokens per second — roughly human reading speed. The feat was made possible not by new hardware but by a memory-hierarchy architecture inspired by Google's Gemma 3n, splitting the model so a small reasoning core sits in RAM while a large embedding table is accessed in tiny slices from flash storage. The project drew coverage from Tom's Hardware, The Register, and others, representing a hundredfold jump from the previous ESP32 parameter record of 260,000. A lesser-noticed follow-up project, p-for-llm, went further on August 5, running a 180.9-million-parameter mixture-of-experts model on the similarly priced ESP32-P4 at around 9 tokens per second, with basic instruction-following and an early tool-calling capability. Together, the two projects mark a significant shift in what edge AI hardware can achieve without any cloud offloading.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer runs 180M-parameter MoE language model on a $10 ESP32-P4 microcontroller

A developer named cyfrit published a project called p-for-llm in early August 2026, demonstrating a 180.9-million-parameter mixture-of-experts language model running inference on an ESP32-P4 microcontroller priced between six and ten dollars. The model uses ternary-weight quantization and top-1 expert routing to keep compute costs low, generating around 9 tokens per second within the chip's limited memory. Notably, cyfrit trained the model on roughly 12 billion tokens using a single consumer RTX 5060 Ti GPU, without institutional backing or a compute cluster. The project carries one significant caveat: weights must currently be loaded onto the board via USB from a host machine at startup, meaning the device is not yet fully self-contained. SD card loading, which would make the board fully independent, is listed by the author as planned future work rather than a completed feature.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Congo's $1.26B Railway Revival Highlights Africa's Colonial Infrastructure Legacy

Congo's Council of Ministers approved a $1.26 billion deal in August 2026 to rehabilitate the Dilolo–Sakania railway, a roughly 1,000-kilometre line connecting the Angolan and Zambian borders through the copper belt. The project forms part of the broader Lobito Corridor, whose backbone is the Benguela Railway, originally chartered in 1902 to move Katanga copper to an Atlantic port. Critics note that most colonial-era African railways were built as extraction routes — connecting mines or plantations directly to coastal ports — rather than as networks linking African cities and markets to each other. Compounding this structural problem, colonial powers built railways in incompatible gauges, meaning freight must often be unloaded and reloaded at modern borders where two legacy networks meet. The rehabilitation raises a broader question about whether Africa is rebuilding the infrastructure it needs or simply restoring a century-old system designed for someone else's benefit.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds xAgent to Make AI Automation Affordable for Small Teams

A developer began building xAgent in April 2025 with the goal of creating a task-oriented AI system capable of running work autonomously. Early experiments with a single-agent setup revealed limitations, prompting a shift to multiple collaborating agents each handling distinct responsibilities. Rising token costs led the developer to run open-source models locally on a modified RTX 4090, though smaller models proved less reliable on complex, long-running tasks. Unable to afford continuous access to top-tier hosted models, the developer focused on building a robust system architecture that could maintain task progress even with less capable models. By August 2025, a working version was ready, and a decision to support multiple users at home shaped xAgent into a server-based, multi-user platform.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Mid-Task Edits Expose a Blind Spot in AI Agent Permission Systems

A developer discovered that standard permission frameworks for AI agents fail to account for task changes made after an agent has already begun work. When the developer edited a task definition mid-execution, the agent continued operating under stale instructions despite having valid permissions. The core issue is that permission levels define what an agent may do, but not what should happen when the underlying task is modified after authority is granted. The proposed fix involves binding a specific task revision to each handoff and requiring the agent to verify that revision before any write operation. If a mismatch is detected, the agent stops and returns a structured result rather than re-planning autonomously, preventing it from self-authorizing actions under a task it was never cleared to execute.

0 comments Read more at DEV Community