Developer runs 125B-parameter AI model on 16GB Mac using SSD streaming trick

·1 views

A developer has released an open-source tool called Slotstream that enables running Qwen3-235B-A22B-Flash (a 125-billion-parameter model normally requiring over 100GB of memory) on Macs with as little as 16GB of RAM. The tool achieves this by using expert-offloading and SSD streaming to compensate for limited onboard memory. Built natively for Apple hardware using MLX and Swift, it can run the model at around 12 tokens per second on a 48GB Mac. Slotstream ships with an auto-mode that automatically balances memory usage against inference speed. The developer plans to add support for speculative decoding via the MTP module in a future update.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Missing PostgreSQL indexes caused React dashboard crash with TypeError in production

A production bug in an internal Condo Dashboard triggered a JavaScript TypeError because the /condos/metrics API endpoint was returning null or a single object instead of an array, crashing a React dropdown component. The root cause was duplicate rows in the broker_tokens table, resulting from missing unique database indexes that allowed multiple rows with the same broker_id and token pair. Initial workarounds — a frontend Array.isArray guard and forced API normalization — suppressed the error but left KPI calculations incorrect and the underlying data inconsistency unresolved. The permanent fix involved adding a unique index on broker_tokens, a performance index on condo_metrics, and a foreign key constraint in apps/api/src/db/db.ts to prevent duplicates at the database level. With the schema corrected, the API controller was also updated to reliably return a clean array, restoring accurate dashboard metrics.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Openship Self-Hosted Deployment Platform Draws Developer Interest on GitHub

Openship, an open-source self-hosted deployment platform by oblien, has recently gained over 111 GitHub stars in a single day, reflecting growing developer interest. The platform allows teams to run deployment infrastructure within their own environment, reducing reliance on external control planes and keeping sensitive metadata in-house. It is particularly suited for workloads involving internal AI services, private build artifacts, or shared deployment credentials. Security best practices recommended for deployment include placing the service behind an internal reverse proxy, enforcing least-privilege token policies, and validating backup and recovery procedures before treating it as a critical dependency. Developers are advised to evaluate Openship as full infrastructure rather than a simple dashboard, carefully assessing its deployment lifecycle, authentication model, and operational resilience.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds AI Agent to Fix Weak Tests, Uncovers 8 Bugs in His Own Tool

A software developer built an AI agent over 30 hours to automatically generate tests that kill surviving mutation testing results, entering the micro1 Frontier Engineering Challenge among roughly 7,800 registrants. The project was motivated by a well-known gap in the industry: line coverage metrics show whether code ran, but not whether tests would catch errors if the code were wrong. While benchmarking 12 popular Python libraries, the developer found that 80 of 133 surviving mutants lived on code never executed by tests at all, not merely weakly tested. Widening test scopes up to 40 times barely changed this figure, suggesting the 'tests that run but don't assert' problem is largely specific to AI-generated tests rather than human-written ones. Before completing the agent itself, the developer discovered eight separate bugs in his own measurement harness, each capable of producing confident but incorrect results.

0 comments Read more at DEV Community

ProgrammingHacker News ·

Claude Fable 5.1 and Claude Mythos 5.1

Article URL: https://www.anthropic.com/claude-fable-and-mythos-5-1 Comments URL: https://news.ycombinator.com/item?id=49525496 Points: 39 # Comments: 3

0 comments Read more at Hacker News