Cómo calcular si un LLM local cabe en la memoria de tu equipo

·5 views

Antes de descargar un modelo para ejecutarlo en local, conviene estimar si cabe en la memoria disponible, sumando los pesos, la caché KV y el overhead. Los pesos dependen del número de parámetros y de la cuantización, y si no se reconoce esta última se debe asumir FP16 como opción pesimista. La caché KV crece linealmente con la longitud del contexto, por lo que un modelo puede caber con 4K tokens pero no con 32K. En GPU dedicadas el límite es la VRAM, mientras que en Apple Silicon solo puede usarse aproximadamente el 75 % de la memoria unificada. Por último, el ancho de banda de memoria dividido entre el tamaño del modelo marca un techo orientativo de la velocidad de generación.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Voice Creators Need Rights Records That Outlast Vendor Roadmaps

Consent given at signup may not protect a creator if a voice AI vendor later changes direction, retires a model, or shuts down. Ownership should be shown over time, not just at onboarding, since platforms that leave creators with dead links, missing usage histories, or unclear payouts have created dependency rather than durable rights. The article argues that consent, permitted uses, term, revocation rules, payout details, and licensing history should live in a record independent of any single vendor's dashboard. It also stresses that withdrawal should stop future availability while keeping valid past grants inspectable, and that new providers or materially new uses should require fresh decisions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

ScriptForge turns plain text into hand-drawn study pages with math and diagrams

A developer built ScriptForge, a browser-based tool that compiles plain text, LaTeX-style math, and small diagram blocks into hand-drawn notebook pages. The creator was frustrated that AI chatbots returned walls of text or blocky charts instead of the page-style layouts they use when studying. ScriptForge renders deterministic handwriting with per-character variation, so the same seed reproduces an identical page. The tool runs in the browser with no account, cloud service, or build step, and exports multi-page PDFs or PNGs. Its diagram blocks can produce items such as free-body diagrams, timing waveforms, and molecular structures.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Tutorial Shows How to Display the Serving Tier Beside AI Drafts for Reviewers

A tutorial argues that AI review tools should display which model tier served a draft, since reviewers cannot judge output they have no evidence about. It uses an example where a reviewer approved a draft that had silently dropped a refund clause without knowing the model or server involved. The guide walks through running a probe to capture serving details, rendering a tier strip with fields such as model, finish reason, latency, and context status, and treating unknown values as valid states. It also describes writing stop conditions and testing reviewer behavior across scenarios like context overflow, rate limits, and mid-task failures. The piece is disclosed as part of an outreach effort for MonkeyCode, an open-source AI coding project, though its claims are presented as items to verify.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Bulk Image Editor Runs Batch Image Pipelines Locally in the Browser

An indie developer describes building Bulk Image Editor, a browser-based tool that processes image batches on the client instead of uploading them to a server. The article argues that folder-level pipelines covering cropping, resizing, compression, format conversion, watermarking, and renaming suit sellers and content teams better than single-purpose tools. It cites privacy for unreleased product photos, avoiding duplicate uploads, and consistent rules across catalogs as key reasons for local processing. The author also notes that focused single-function pages attract new users while the multi-step Batch Editor supports repeat production work.

0 comments Read more at DEV Community