Why AI Models Need Cloud Servers: RAM Limits, VRAM, and Quantization Explained

·3 views

Running large AI models locally is constrained by fundamental hardware limits, not software choices. A 7-billion-parameter model requires roughly 28 GB of storage at full precision — more than most consumer graphics cards can hold — forcing a trade-off between model size and output speed. When model data spills beyond a GPU's VRAM, it must repeatedly cross a much slower CPU-to-GPU connection, dropping inference speed from around 70 words per second to just 4. A technique called quantization reduces memory needs by storing numbers with less precision, shrinking a 28 GB model to around 7 GB at the cost of some accuracy. Larger 70-billion-parameter models require 140 GB or more even when compressed, making cloud-based APIs a physical necessity rather than merely a commercial preference.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Platform Teams Risk Becoming the Same Bottleneck They Were Built to Replace

Platform engineering was introduced to reduce developer cognitive load and enable self-service, but critics argue many teams have recreated the old ops silo under a new name. According to the State of Platform Engineering Report Volume 4, 45.3% of platform teams cite developer adoption as their biggest challenge, with 36.6% relying on top-down mandates rather than organic uptake. Backstage, Spotify's internal developer portal solution, holds roughly 89% market share among IDP adopters, yet analysts warn that polished portals often mask manual approval processes rather than eliminating them. The same report found that 78% of platforms attempting to solve all developer problems at launch eventually fail, while 29.6% of platform teams track no success metrics at all. Experts including Honeycomb CTO Charity Majors argue that genuine platforms should offer guardrails rather than gatekeeping, warning that heavy abstractions prevent developers from diagnosing and resolving their own incidents.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Laractions Package Moves Laravel Business Logic Out of Controllers into Action Classes

Laractions is a Composer package for Laravel that extracts business logic from bloated controller methods into dedicated, single-purpose action classes. Controllers often accumulate responsibilities over time — such as database writes, emails, and logging — that should not live alongside request-handling code. The package provides an Artisan generator to scaffold action classes under App\Actions, with dependency injection supported via Laravel's service container. A refactored controller simply validates the request and delegates to the action using a fluent run() method, which forwards arguments to the action's handle() method. The package also supports model-bound actions, allowing developers to scope operations like cancellations directly to an Eloquent model instance.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Larakeep Package Brings Dedicated Keeper Classes for Laravel Model Field Computation

A new open-source Laravel package called Larakeep introduces a structured way to handle derived model fields such as computed totals, slugs, and cached counts. Developers install it via a single Composer command, after which a service provider auto-registers with no additional configuration needed. Each 'Keeper' is a plain PHP class containing formula methods for specific model columns, keeping computation logic separate from observers and model files. Models are linked to their Keeper classes using a HasKeepers trait and a KeptBy attribute, then fields are populated in memory by calling a process() method, which can be chained with save() to persist changes. The package also supports parameterized methods and custom verb prefixes, giving developers flexible control over when and how derived fields are calculated.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Angular 22 Introduces WebMCP to Expose App Capabilities to Browser AI Agents

Angular 22 introduces an experimental feature called WebMCP, which allows AI agents running in the browser to discover and interact directly with an Angular app's services and signals through a typed interface. The feature is built on the W3C ModelContext API, a draft browser standard that registers tools on navigator.modelContext for compatible agents to query and call. As of June 2026, Edge 147 supports the standard natively, Chrome 149 has an active Origin Trial, and Firefox and Safari have committed to support by Q3 and Q4 2026 respectively, with broad adoption expected around mid-2027. Unlike Agent Skills, which enables an Angular app to call out to AI, WebMCP works in the opposite direction — letting AI agents drive the app's functionality. The feature ships as experimental under provideExperimentalWebMcpTools() with no stability guarantees, as the underlying W3C specification is still evolving.

0 comments Read more at DEV Community