Developers Can Now Run Google Gemma 4 AI Agents Locally Using Docker Model Runner

A developer has demonstrated how to run an AI agent entirely on a local machine by replacing the cloud-based Gemini 2.5 Flash model with Google's open-weight Gemma 4, served via Docker Model Runner. Docker Model Runner is a built-in Docker Desktop feature that pulls and runs large language models locally, exposing an OpenAI-compatible API on localhost without requiring cloud credentials or API keys. The setup uses Google's Agent Development Kit (ADK) alongside BigQuery MCP tools, with only the model endpoint needing to change from the original cloud deployment. Gemma 4 is available in multiple sizes on Docker Hub, ranging from a 2.94 GiB edge-friendly variant to a 20 GiB high-quality dense model, with the E4B variant recommended as the best balance of speed and quality for local agent development. The approach eliminates per-call API costs and latency during development while keeping the agent logic, tools, and system instructions identical to the production cloud version.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in