Developer measures 90x display lag in local LLM streaming, finds Ollama update fixes issue

·1 views

A developer building a local LLM coding IDE observed that allocating all CPU cores to the model made the application feel slower despite similar token generation speeds. Testing on Ollama version 0.21.0 revealed a 90x increase in display lag and a 6x slower UI response when using six cores versus four. The core issue was CPU saturation by the LLM, which starved the renderer and disrupted the user interface. The problem was resolved in Ollama version 0.40.2, where default thread management changed to avoid full CPU saturation, eliminating the display lag.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer creates 'LORE WALK' app that uses on-device AI to generate stories from photos

Penta Dank of Next Realm Interactive has developed a mobile web app called LORE WALK. The app encourages users to take walks and photograph interesting objects they encounter. An AI model running entirely on the user's device then generates short mythical lore pieces about these photographed objects. The resulting 'Lore Shards' are compiled into a 'Lore Codex' that can be exported as Markdown or an image. The app was created to help the developer's creative director engage more with the physical world.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

RTK vs PPK Drone Positioning: Choosing the Right Method for High Accuracy

Professional drone mapping uses RTK or PPK systems to achieve centimeter-level positioning accuracy. RTK provides real-time corrections during flight but requires a continuous data link via cellular or radio. PPK records raw data for processing after the flight, eliminating the need for a live link and allowing operation in areas with poor connectivity. Many professional crews use both systems simultaneously, relying on RTK for live feedback and PPK as a reliable backup. The choice depends on the need for real-time results versus operational flexibility in challenging environments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer releases open-source TypeScript framework for customizable AI agents

A developer has released Chloe, an open-source TypeScript framework for building AI agents. It allows developers to combine traditional code with AI steps for business applications. The framework emphasizes control, letting developers own the code, tools, and tests, and run agents on their own infrastructure. Chloe was shaped by the developer's two years of experience as a forward-deployed engineer working with business needs. The project is free and aims to make creating reliable, understandable agents easier.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Guide: Core Regex Syntax for 80% of Common Tasks

A new guide from DEV Community focuses on the essential 20% of regular expression syntax that handles most practical development work. It emphasizes that developers do not need to master complex patterns, but rather a core set of anchors, character classes, quantifiers, groups, and alternation. The guide provides production-ready patterns and highlights critical pitfalls like greedy-vs-lazy confusion and catastrophic backtracking. It covers the common syntax subset shared by major programming languages like JavaScript, Python, and Java.

0 comments Read more at DEV Community