Qwen 3.8 27B 1-bit Quantization Delivers Under 1 Token/sec on RTX 4090

·1 views

A developer benchmarked Qwen 3.8 27B in both 4-bit and 1-bit quantization formats on an NVIDIA RTX 4090 and a MacBook Pro M1 Max to evaluate real-world inference speeds for local AI agent use cases. The 4-bit (q4_K_M) model consumed around 9.5GB of VRAM and averaged 28.5 tokens per second, making it practical for agent workflows requiring fast responses. By contrast, the 1-bit (q1_K) model used only 4.5GB of VRAM but averaged a mere 0.8 tokens per second — effectively unusable for any interactive or automated task. Testing was conducted using Ollama 0.1.37, with each configuration run 10 times on a standardized coding prompt to ensure consistent results. The findings challenge the assumption that lower VRAM usage translates to faster inference, highlighting that aggressive quantization can severely degrade throughput despite memory savings.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

SQL Join Types Explained Using a Small Hospital Database in PostgreSQL

A tutorial published on DEV Community uses a practice hospital database with 10 patients, 10 doctors, and 10 appointments to explain how SQL joins work in PostgreSQL. The database intentionally omits foreign key constraints to illustrate that joins operate purely on matching values at query time, independent of referential integrity rules. Two patients and three doctors with no appointment records serve as test cases to demonstrate how different join types handle unmatched rows. An INNER JOIN returns only the 10 matched appointment rows, silently excluding the five people with no counterpart on the other side. A LEFT JOIN returns 12 rows, preserving all patients including the two without appointments by filling unmatched columns with NULL values.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Open-Source Python Dashboard Monitors VoIP Call Quality via Telnyx Webhooks

A developer at Telnyx has released an open-source call quality monitoring tool built with Python and Flask that processes real-time voice metrics including MOS, jitter, latency, and packet loss. The application receives Telnyx webhooks, verifies each payload using Ed25519 signature validation via pynacl, and stores metrics in SQLite for historical analysis. A live browser dashboard is updated through Server-Sent Events, and alerts are triggered automatically when quality thresholds are breached. The project includes a demo server that generates its own keypair and simulates webhook payloads locally, requiring no Telnyx account or external services to run. The source code is publicly available on GitHub under the Telnyx code examples repository.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Web Share API Lets Browsers Replace Outdated Social Share Icon Rows

Most websites display a fixed row of social media share icons that were chosen years ago and do not reflect the apps actually installed on a reader's device. This approach requires ongoing maintenance as platforms rebrand or emerge, and some share-URL patterns can silently pass user data to third-party sites. The Web Share API offers a three-line JavaScript alternative that triggers the device's native share sheet, showing only the apps the user genuinely has installed. It works on HTTPS pages when called from a direct user interaction, and supports sharing not just URLs but also locally generated files such as images or PDFs. Despite broad browser support, many developers have been slow to adopt it, largely due to its historically inconsistent implementation across platforms.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Serverless Chrome Extension to Export Facebook Group Data Locally

A freelance developer has released FB Extractor, a Manifest V3 Chrome extension designed to pull member and group data from Facebook without relying on any external servers or APIs. The tool runs entirely within the browser, storing all extracted data on the user's own device, and exports it as CSV or XLSX files ready for use in Excel or a CRM. To reduce the risk of account bans, the extension simulates human-like scrolling and randomized delays between actions. By design, the extension does not collect contact details such as emails or phone numbers, limiting extraction to public identifiers like user ID, name, and profile picture. A free tier allows up to 50 groups and 200 members per export run, with a paid Pro version offering unlimited extraction.

0 comments Read more at DEV Community