SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Running Qwen3.8-Flash-Next Locally Depends on Total Memory, Not Active Parameters

0
·1 views

Qwen3.8-Flash-Next activates only 6B parameters per token, but its GGUF files range from about 66 GB for the smallest 1-bit quant to 111 GB for the common UD-Q4_K_XL. The article argues that total memory, not the card's active parameter count, determines whether the model can run locally. A community llama.cpp benchmark reported 39 tok/s for UD-IQ4_XS with experts held in 96 GB of system RAM and a simulated 24 GB VRAM budget. The n-gram embedding table, about 27 GiB in one quant, can sit in system RAM or be paged from SSD because each token reads only a few rows. Qwen's release is an experimental preview under the qwen-community-1.0 license, which is not Apache 2.0.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Agent-Written Endpoint Triggers N+1 Queries; Query-Count Test Catches It Early

A developer describes an AI coding agent's GET /orders handler that returned correct data but issued one database query per order, 101 queries for a 100-row page. Existing functional tests passed because they only checked the response body, not the query count. The author shows how the N+1 pattern arises in node-postgres and Express code, even though each line looks correct. To catch it, the author proposes a test helper that wraps the pool's query method to record each SQL statement while still executing it. Running this counter within a test lets a branch fail before the regression is merged.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Vietnam and South Korea to Hold AI and Semiconductor Conference in Hanoi on November 12-13

Vietnam's Ministry of Science and Technology has announced the First Vietnam-Korea AI and Semiconductor Conference, scheduled for November 12-13, 2026 in Hanoi. The summit is expected to draw more than 400 delegates, 50 semiconductor and AI companies, research institutes, and universities from both countries. Organizers include the ministry's Department of Digital Technology and AI, FPT Corporation, the Korea AI-Semiconductor Fabless Alliance, and VINASA's semiconductor committee. Discussions will focus on aligning bilateral policy, integrating semiconductor supply chains, and building workforce programs tied to Vietnam's national semiconductor strategy and its initiative to train 50,000 semiconductor and AI engineers.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Searchable Incident Archive Lets a Platform Engineer Track 938 Write-Ups

A developer documenting every Kubernetes platform bug with symptom, root cause, fix, and proving test has built up roughly 1,750 documents, including 938 incident write-ups. To make them useful, they built a pgvector-backed search index on a multi-cluster setup, deployed via ArgoCD, with an indexer that embeds new and changed docs and prunes deleted ones. The index feeds a pre-filing duplicate check and a Slack question-answering command. A Grafana dashboard tracks index freshness, drift, backlog, run health, and reachability so the team can trust the index's answers. A recency-related answer failure was traced to pure similarity ranking, and the fix was to detect recency terms, widen the candidate pool, and sort newest first.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Vietnam-Korea AI and semiconductor conference set for Hanoi on November 12-13

Vietnam's Ministry of Science and Technology has announced the first Vietnam-Korea Artificial Intelligence and Semiconductor Summit (VKAISS 2026), scheduled for November 12-13 in Hanoi. The event will bring together more than 400 delegates, over 50 leading technology and chip companies, and universities and research institutes from both countries. It is co-organized with FPT Corporation, the Korean AI and Fabless Semiconductor Association, and VINASA's semiconductor committee, with participation from Korea's national nano fabrication center. Discussions will focus on linking semiconductor and AI supply chains and on training high-quality talent under Vietnam's national semiconductor strategy and its plan to develop 50,000 chip and AI engineers. Separately, the Vietnam-Japan AI Hackathon 2026 concluded in Tokyo, where a team won first prize for an autonomous AI agent that predicts faults and optimizes maintenance for offshore wind farms.

0 comments Read more at DEV Community