80B Qwen AI Model Now Runs Locally on 4.3GB RAM Using Advanced Compression

·1 views

Developers in 2026 demonstrated that an 80B-parameter Qwen language model can run on just 4.3GB of RAM on consumer Apple hardware, sparking widespread attention online. The feat is achieved through a combination of techniques including structural pruning, group-wise weight sharing, and dictionary coding, reducing the model to roughly 25 billion live parameters before further compression. Apple's M5 Max chip plays a central role, offering over 900GB/s memory bandwidth and a sparse-matrix-aware AMX-3 coprocessor that sustains around 4–5 tokens per second locally. A 35B model was separately reported running on a base iPhone 18 Pro, entirely on-device without cloud connectivity. These developments mark a significant shift in edge AI, making large language models practical for private, battery-powered, offline use on mainstream consumer devices.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Open-Source CRM 'trycompai' Lets AI Agents Autonomously Manage Sales Workflows

A new open-source CRM called trycompai/crm has been built with an agentic-first design, meaning AI agents can autonomously enrich leads, draft emails, route prospects, and execute workflows without constant human input. Unlike traditional CRMs that rely on manual data entry, this system treats AI agents as core participants rather than add-on features. Built with TypeScript, Node.js, and a React frontend on a microservices architecture, it offers REST and GraphQL APIs alongside a real-time event bus for agent communication. The platform includes configurable approval thresholds to keep human oversight in place for high-stakes decisions, as well as a full agent activity feed for auditability. Positioned as a privacy-friendly alternative to proprietary tools like Salesforce and HubSpot, it allows organizations to self-host and fully customize agent behavior without vendor licensing fees.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Open-Source OCR Library decimen-optical-transfer Targets High-Precision Number Extraction

A GitHub user named bashalarmistalt has developed decimen-optical-transfer, an open-source OCR library designed specifically to extract numerical data from images, documents, and handwritten tables. Unlike general-purpose OCR tools such as Tesseract or EasyOCR, this library focuses exclusively on digits, decimal points, negative signs, and scientific notation. It uses a character-level attention mechanism and a custom preprocessing pipeline to reduce common misreads between visually similar characters like 0 and O or 1 and l. The tool also includes a layout parser for identifying table structures, making it useful for financial statements, invoices, and scientific reports. Models are packaged as ONNX Runtime files for CPU use, with an optional PyTorch backend available for faster GPU-based batch processing.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

LiDAR Data Reveals 3 Million People Once Lived in Just 3% of Amazon

New research using LiDAR remote-sensing technology suggests that up to 3 million people inhabited the Amazon rainforest before European contact, concentrated in just 3% of the surveyed forest area. A consortium of archaeological and remote-sensing teams analyzed over 20,000 square kilometers of LiDAR data spanning Brazil, Bolivia, and Ecuador to reach this estimate. The technology works by firing laser pulses from aircraft to penetrate the forest canopy and build detailed 3D terrain maps, revealing ancient causeways, plazas, terraces, and irrigation canals invisible to ground-based surveyors. Rather than scattered settlements, the data shows pre-Columbian populations clustered densely in specific zones, challenging long-held assumptions about how indigenous Amazonians organized their societies. The findings were further validated through targeted ground excavations, combining computational tools like machine learning with traditional archaeological fieldwork.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

NHS apologizes after Palantir staff accessed identifiable UK patient data without consent

The UK's National Health Service has formally apologized after admitting that staff at data analytics firm Palantir Technologies were granted access to identifiable patient records, contrary to prior public assurances. Palantir was awarded a £330 million contract in 2023 to build a Federated Data Platform intended to operate on pseudonymized data only. A whistleblower exposed internal emails indicating Palantir engineers had viewed live patient records using so-called 'break-glass' emergency access for routine debugging and data integrity checks. The NHS acknowledged the lapse following an internal review and has suspended all non-essential Palantir access pending a full audit. The UK's Information Commissioner's Office has launched a formal investigation into the potential data protection breach.

0 comments Read more at DEV Community