Sizing Ollama models for CPU-only servers: a RAM guide

·2 views

Ollama can run models on a CPU alone by loading them into system RAM, so the key question is which model fits the available memory with room left for context and the OS. The guide lists download sizes for common model tags, from about 292 MB for gemma3:270m up to 17 GB for gemma3:27b. Its rule of thumb is to keep the model file at or under roughly half the server's RAM at default context, then verify. It also covers quantization differences, such as llama3.2:1b defaulting to 8-bit while its 4-bit version is 808 MB. Four settings, including OLLAMA_CONTEXT_LENGTH and OLLAMA_NUM_PARALLEL, affect how much extra memory is used, and ollama ps and free -h help confirm the result.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

FAA Begins Retiring Decades-Old Floppy Disks From Air Traffic Control Systems

The Federal Aviation Administration has started phasing out floppy disks that have been used in air traffic control operations for decades. Announced by Transportation Secretary Sean P. Duffy, the move is part of a broader effort to modernize aging agency technology. The article provided is only a link to an FAA newsroom release, so details such as the exact timeline, number of systems affected, and replacement technology are not available here. The story was shared on Hacker News, where it drew modest engagement.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Unchanged 39-of-40 test score hid a shift in which CSV parser case failed

An AI developer built 40 CSV fixtures with literal expected results and ran them through the csv-parse 6.1.0 library via a custom policy adapter. Under default settings one quote-handling case disagreed with its expectation, and relaxing quote handling fixed that case but broke another, so the score stayed at 39 of 40. The author kept the expectations fixed and documented both mismatches, arguing that matching pass totals can conceal a changed behavior contract. The piece recommends recording file acceptance or rejection, exact returned values such as leading-zero identifiers, and parser configuration separately from application import policy. It also notes that the injected-fault and cross-implementation checks give only partial assurance and do not cover production databases or security.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Social Engineering Exploits Human Psychology and How to Defend Against It

Social engineering is a cybersecurity threat that targets people rather than software, using psychological manipulation to get victims to reveal confidential information or take harmful actions. Attackers often gather information about their targets first to make their deceptions more believable. Common tactics include creating false emergencies that pressure victims into acting quickly, and pretexting, where the attacker impersonates a trusted authority such as IT staff or an executive. Physical methods like baiting, for example leaving infected USB drives in workplaces, also exploit curiosity to bypass technical defenses. The article uses concrete workplace scenarios to show how these manipulations operate and how individuals can recognize them.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why a Typeahead Trie Should Store the Top 10 Completions at Every Node

A system design interview question about search autocomplete asks what a service does when a user types a single character. The naive approach of collecting and sorting every query under a prefix's subtree on each keystroke is correct but too slow at scale. Latency limits of about 0.1 seconds, including network time, make precomputation necessary. Illustrative capacity estimates show reads vastly outnumber writes and that client-side debouncing and caching can cut request volume roughly fourfold. The recommended fix is to precompute the top 10 completions at every trie node during build time, so each lookup only walks the prefix and returns a ready list.

0 comments Read more at DEV Community