How to Diagnose and Fix Ollama Ignoring Your GPU on Linux and Windows

·1 views

Ollama, a local AI model runner, may default to CPU inference even when a GPU is available, significantly slowing performance. Running 'ollama ps' while a model is loaded reveals the PROCESSOR column, which shows whether inference is running on the GPU, CPU, or split between both. The most common causes of GPU detection failure include outdated drivers, missing user group memberships on AMD Linux setups, a pinned OLLAMA_LLM_LIBRARY environment variable, or containers launched without GPU access flags. Partial GPU offloading typically occurs when model weights and context cache exceed available VRAM, and can be addressed by using smaller quantizations or reducing context window size. Checking the Ollama server log for the 'inference compute' line helps pinpoint exactly which hardware was detected at startup.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer builds Polish-keyword programming language, gains deep Ruby insights

A Ruby developer spent 18 months building AlexScript, a fully featured interpreted scripting language whose keywords are written in Polish, his first language. The project grew from a weekend experiment into a language with a standard library, async/await support, a debugger, and a self-hosted web framework. Building the interpreter revealed several Ruby performance nuances, including that Ruby's throw/catch is far more efficient than raise/rescue for non-local exits like function returns. The developer also discovered that scanning UTF-8 strings character by character causes accidentally quadratic performance, fixable by using getbyte and byteslice for O(1) byte access. A unified method dispatch table — rather than separate registries for native and user-defined methods — further simplified the interpreter's architecture and improved lookup efficiency.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Go developer builds linter to enforce file-level visibility in flat packages

A Go developer has released declscope, an open-source linter designed to enforce file-scoped access boundaries within Go's flat package structure. Go conventionally offers only two visibility levels — exported and unexported — with no native file-scope option, meaning any unexported symbol is accessible across all files in a package. The tool addresses a growing concern that AI coding agents, finding unexported helpers in scope, call or modify them freely since such cross-file conventions exist only as informal comments rather than compiler-enforced rules. The developer noted that even explicit instructions in AI configuration files failed to reliably prevent boundary violations, prompting a machine-checkable solution. declscope can be installed via standard Go tooling and flags violations when declarations are used outside their intended file scope, providing agents and reviewers with clear repair guidance.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Agents Can Use Far More Energy Than Single Prompts, Researcher Finds

Climate scientist Zeke Hausfather analyzed the energy consumption of AI agentic workflows, finding they can use significantly more electricity than standard one-off chatbot prompts. Unlike a simple question-and-answer exchange, AI agents repeatedly re-process their entire accumulated context with each step — such as running commands, reading files, or calling tools — driving up computational costs. Hausfather estimated his own daily AI agent usage consumed between 1.2 and 5.9 kWh, with visible output accounting for just 0.4 percent of total tokens processed. The concern is not one user's consumption in isolation, but the aggregate energy demand when many users run agentic workflows simultaneously. While an exact global climate impact remains uncertain, the directional finding is clear: agentic AI systems can carry a substantially larger energy footprint than most users assume.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

ECC Agent Harness Runs in Cursor Without Context Bloat Using Modular Setup

Developer affaan-m released ECC v2.0, an agent harness that standardizes AI coding rules, skills, and MCP configurations across tools like Cursor, Claude Code, and Codex. A key challenge when using ECC inside Cursor is context bloat, where loading all instinct files and tool schemas at once rapidly consumes tokens and slows multi-turn sessions. The recommended fix is modular rule scoping — symlinking only language-specific rules relevant to the active workspace rather than loading the full ECC repository into the root prompt. MCP servers are added selectively via a project-level config file to further limit unnecessary context. Testing showed that routing Cursor through a prompt-caching gateway endpoint reduced multi-turn context costs by roughly 80–90% while preserving reasoning quality and chat history.

0 comments Read more at DEV Community