Edge ML: Quantization, Pruning and Latency Trade-offs on Microcontrollers

·1 views

Running machine learning models on microcontrollers and low-power embedded boards is constrained by tight limits on SRAM and flash storage. Because dynamic allocation is often avoided, the full activation arena must fit in SRAM ahead of time, so shrinking weights alone does not guarantee a model will run. The article explains quantization, which maps 32-bit floats to 8-bit integers using a scale factor and zero-point, in two forms: post-training quantization, which is simple but can lose accuracy, and quantization-aware training, which typically recovers that loss. It also includes a TensorFlow Lite for Microcontrollers example for producing a full-integer INT8 model.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

GreenBuddy AI Suggests Outdoor Activities Using a Local Gemma Model

A developer has built GreenBuddy AI, a small Python and Streamlit app that recommends outdoor activities to users. Suggestions are tailored to available time, surroundings, interests, and energy level, covering options like gardening, birdwatching, and short walks. The app runs the open-weight Gemma model locally through Ollama, so user preferences are processed on the user's own computer instead of a hosted API. The developer says the aim is to help people spend less time on screens, and plans future additions such as an activity journal and seasonal gardening tips.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

One-Call Text-to-SQL System Scores 77.1% on BIRD Dev Set

A developer ran the schemagate system on all 1,534 BIRD dev questions and reported 77.12% execution accuracy using Claude Opus 5.5. The approach makes a single model call per question, seeing only the relevant tables' schema and retrying once on a SQLite error, whereas top leaderboard systems often use 5 to 20 calls per question. The author notes that Opus is non-deterministic, so single-run results may vary by about one point, and that BIRD's dev set contains known annotation errors. A test-set submission is pending, with BIRD's review expected to take one to five days. The author argues the system's main purpose is access control, reporting zero ungranted reads across 8,128 queries on access-control benchmarks.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Rust Closures Work: Compiler-Generated Types and Capture Traits

The article explains that Rust closures were introduced because standard function pointers cannot capture variables from their surrounding scope. It describes closures as compiler-generated, anonymous types that store captured variables in internal fields. The compiler implements the Fn, FnMut, or FnOnce trait depending on whether captured values are borrowed immutably, borrowed mutably, or consumed. The author notes that the move keyword is often needed to satisfy lifetime rules, such as when spawning threads or returning closures from functions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

SlothSort: Local-First AI File Organizer That Asks Before Moving Files

Developer Tanmay Dalvi built SlothSort, an AI-powered tool that scans a chosen folder and proposes how its files should be sorted into destination folders and renamed. Unlike fully autonomous agents, the AI only suggests a plan, which deterministic code validates before the user reviews, edits, and approves it. The tool supports local inference through Ollama, an optional hosted Google Gemini provider selected explicitly, collision detection, and undo for eligible operations. Its design avoids any file deletion in the organization workflow, and it includes a demo mode using a sample folder. The author notes that privacy depends on the chosen provider, since filenames can reveal sensitive personal information.

0 comments Read more at DEV Community