Google's Gemma 4 AI Model Runs on a 4 GB Laptop GPU Using QAT Compression

·1 views

A developer has demonstrated that Google's Gemma 4 E2B language model can run on a 2021 Lenovo Yoga 9 laptop equipped with a 4 GB GTX 1650 Ti GPU, using a Quantization-Aware Training (QAT) checkpoint. The model's standard bfloat16 format requires 9.5 GiB of memory, far exceeding the GPU's 4 GB ceiling, but the QAT version reduces the GPU-resident portion to just 1.31 GiB. Unlike standard post-hoc quantization, QAT trains the model with 4-bit storage in mind from the outset, preserving quality while dramatically cutting memory use. The compressed model loaded in 1,618 MiB, left over half the GPU free, and achieved a decode speed of 73.75 tokens per second. The developer also published a suite of Python MCP tools on GitHub to simplify managing the llama.cpp-based local deployment.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

World Models Move Beyond Robotics Into Real-Time AI Video Generation

World models, a concept in AI research describing systems that learn how environments change over time, are gaining renewed attention beyond their traditional use in robotics and self-driving vehicles. Unlike large language models, which predict sequences of text tokens, world models learn causal and physical patterns by processing video and simulated interactions. AI researcher Yann LeCun has described this approach as analogous to the internal mental maps humans build through repeated experience and cause-and-effect reasoning. Video generation company Runway has now applied this technology in its GWM Worlds 2 model, which allows users to steer and shape a video environment in real time rather than simply prompting a finished output. The development signals a broader shift in generative AI toward systems capable of understanding and simulating physical reality, not just language.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How SaaS Teams Should Own OTP Templates for Secure Password Resets

For B2B SaaS password-reset flows, security experts recommend keeping OTP template logic and challenge state within the application itself rather than delegating it to third-party delivery vendors. Both SMS and email are viable channels, but the choice should be treated as a delivery adapter decision, not a security one — the application must control code lifetime, content, and rate limits. A single, atomically consumed challenge record is advised to prevent race conditions where multiple delivery channels could each hold a valid code simultaneously. Template ownership is framed as a core security boundary, since the message content itself can be a vector for user confusion or attacker exploitation. Retention policies for challenge logs should be made configurable to accommodate differing US and EU legal requirements, and reset flows must require a previously verified destination address.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Free macOS App to Track and Identify AI Coding Agent Processes

A developer frustrated by the inability to distinguish AI coding agent processes in macOS Activity Monitor has built Corral, a free open-source native app. Corral identifies and displays processes from tools like Claude Code, Cursor, Codex, Kiro, and Antigravity, showing which project each belongs to, how long it has been running, and its resource usage. The app is built with Swift and SwiftUI, carries an MIT license, requires no special permissions, and makes no network calls. Because process name matching proved unreliable — agents often run under generic names like 'node' — the developer used install directory paths as the stable identifier instead. Each supported tool stores session state differently, requiring custom parsing logic for log files, rollout files, and SQLite databases to surface meaningful session information.

0 comments Read more at DEV Community

ProgrammingGitHub Blog ·

GitHub Copilot App Lets Beginners Review Code, Terminal, and Browser Side by Side

GitHub has published a beginner-focused guide on using the GitHub Copilot app more efficiently. The tutorial addresses a common workflow challenge: reviewing AI-generated code typically requires switching between multiple tabs or windows. The guide walks users through how to view code diffs, execute terminal commands, and preview web applications within a single interface. The side-by-side layout is designed to streamline the process of checking and validating agent-generated code.

0 comments Read more at GitHub Blog