Qwen 3.8 27B Local Setup: GGUF Sizes, KV Cache Savings, and a Critical Flag

·1 views

Alibaba released Qwen 3.8 27B under an Apache 2.0 licence on 13 August, making it freely usable for local deployment, while a separate 2.4-trillion-parameter MoE variant launched under a more restrictive licence. The model uses 64 layers but reserves a KV cache for only 16 full-attention layers, cutting per-token cache cost to 64 KB versus 256 KB on a conventional dense model, making long contexts far more memory-efficient. Quantised GGUF versions range from a 9 GB 2-bit build to a 53.8 GB BF16 file, with the Q4_K_M quant at around 17 GB considered the practical sweet spot for 24 GB GPUs. The model also natively handles images and video without additional wrappers, and ships a separate 0.9 GB vision encoder file. Users running llama.cpp must include the --jinja flag to load the model's custom chat template, as omitting it causes erratic output that is frequently mistaken for a corrupted download.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Google cuts Gemini 2.5 Flash token costs by 50%, boosts coding and automation

Google has released Gemini 2.5 Flash, a developer-focused AI model with improved performance in coding, automation, and multi-step workflows. The new model is priced at $0.75 per million input tokens and $3.75 per million output tokens, roughly half the cost of its predecessor. The release comes just three weeks after the previous version, reflecting Google's strategy of rapid iteration for its Flash series. Benchmark results show notable gains in debugging, code generation, and autonomous agent tasks compared to version 3.6. Google says the price reduction is intended to make large-scale production deployments more accessible for businesses of all sizes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Referral Traffic Hits 1.08% of Web Visits but Understates Broader Influence

A Conductor report analyzing 13,770 domains across 10 industries found that AI-driven referral traffic accounted for 1.08% of total website visits between May and September 2025. The channel grew at roughly 1% month over month during the study period, signaling steady but modest expansion. ChatGPT dominated AI referrals, generating approximately 87.4% of that traffic, making overall figures heavily dependent on a single platform. Experts caution that referral data alone understates AI's true impact, as users may encounter brands in AI-generated answers without ever clicking through to a website. Industry context also matters significantly, with IT sectors seeing AI referral shares as high as 2.8%, well above the cross-industry average.

0 comments Read more at DEV Community

ProgrammingHacker News ·

Study finds coffee consumption linked to metabolic health and sex hormone levels

A new study from the University of Oulu has found connections between coffee consumption and both metabolic health and sex hormone levels. The research suggests that drinking coffee may have measurable effects on hormonal and metabolic markers in the body. The findings add to a growing body of evidence exploring how dietary habits, particularly coffee intake, influence physiological processes. Details of the study were published by the University of Oulu, a Finnish research institution known for health and population studies.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

How to Use Free AI Models for Adversarial API Testing Before Launch

Developers can use free large language models as a low-cost adversarial testing tool by pointing them at API endpoints and prompting them to generate inputs designed to cause failures. Unlike traditional fuzzers, language models can produce semantically plausible payloads that target validation logic and error handling in ways manual testing often misses. A short Python script using any OpenAI-compatible endpoint can automate this process, sending hostile request bodies to a local service and logging status codes and timeouts. Common results include verbose HTTP 500 errors revealing stack traces, and requests that expose unintended error messages — both useful signals before a service reaches production. The approach is positioned as a practical first line of defense for small services lacking formal security audits, not a substitute for professional penetration testing.

0 comments Read more at DEV Community