Gemma 4 31B Uncensored Runs Locally on a 12GB GPU with Vision and 256K Context

A guide details how to run the 31-billion-parameter Gemma 4 31B-it ARA Heretic model on a consumer 12GB GPU using llama.cpp. The approach combines importance-matrix GGUF quantization, dynamic memory fitting, and Flash Attention to avoid out-of-memory failures. The model's multimodal projector enables local image understanding, and its context window can scale to 256K tokens given enough memory. Once served on a local port, the OpenAI-compatible endpoint can connect to coding and agent tools, keeping data on local hardware. The author argues this setup offers an alternative to smaller local models that lose coherence on long contexts and to cloud APIs that charge per token and refuse some security and creative tasks.