NVFP4 Decodes 2.3x Faster Than BF16 on a 16 GB Ada GPU, Matching Its GSM8K Accuracy

·1 views

A developer benchmarked the Nemotron Labs Dory 1.0 4B model in BF16, NVFP4, MXFP8, FP8 and GGUF formats on an RTX 4070 Ti SUPER with 16 GB of memory. Among the vLLM formats, text generation speed followed weight size, with NVFP4 running 2.3 times faster than BF16 through a weight-only kernel. FP8 led on long-prompt reading because it is the only tested format whose multiplications Ada GPUs perform in 8-bit precision. On the full GSM8K math benchmark, all formats scored within a statistical tie with BF16, though NVFP4 matched BF16's next-token choices less often than FP8 or MXFP8. The author also documented several measurement pitfalls encountered along the way, and published the full results and scripts online.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

TrustIntern: Student-built tool flags internship scams with evidence-backed risk scores

A developer built TrustIntern, a tool that helps students check internship offers for signs of fraud. Users paste in a recruiter's email, company details and the offer message, and can attach the offer letter as a PDF or screenshot. The tool runs fixed checks such as email domain matching, domain age, and the wording of the message, alongside research agents that look into the company and public complaints. Each risk score is broken down into specific, traceable pieces of evidence, and reports state they are heuristic indicators rather than proof of fraud. The student version is free, while colleges, job boards and paid postmortem reports for scam victims are planned revenue sources.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

DeepSeek Releases Cheaper, Faster Open-Weight Coding Model V4.1-Flash-1017

DeepSeek released DeepSeek-V4.1-Flash-1017 on October 17, 2026, a minor update to its V4.1-Flash model aimed at agentic coding work. According to vendor-reported benchmarks, the model posts higher scores than its predecessor, completes tasks faster, and produces shorter outputs, while offering a 1M-token context window and MIT-licensed open weights. It leads several agentic coding benchmarks but trails frontier models like Claude Opus 5 on harder reasoning tests, and no independent evaluation exists yet. DeepSeek's API pricing is substantially lower than closed frontier models, with cache hits costing far less than cache misses and off-peak rates at half the peak level. The company also opened its Search & Fetch APIs to all users at $0.75 per 1,000 successful calls.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

NexTrace Uses FalkorDB Graph Database to Detect and Quarantine Cloud Attack Paths

A developer built NexTrace, an AI security agent for the FalkorDB Hackathon, to address cloud breaches where attackers chain small flaws through internal permissions. Traditional tools only send alerts without revealing attack routes or fixing them, and many AI chatbots can only offer written advice. NexTrace models cloud services and their connections as a graph in FalkorDB, allowing it to map the blast radius and find the shortest attack path to sensitive data. The agent can then take direct action by quarantining the vulnerable service and severing risky connections, before verifying that no path to the target remains.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why AI Prototypes Fail in Production and What Reliable Deployment Requires

An AI application that performs well in a demo can still fail once deployed, because the model is only one part of a larger software system. Real-world conditions such as slow providers, traffic spikes, dependency conflicts, and database limits expose problems unrelated to model quality. The article argues that engineers must address containerization, failure boundaries, retries, observability, data security, and recovery from interruptions. It uses a document intelligence service as an example, showing how many external dependencies each add potential points of failure. The author's book, The AI Engineer Volume II, covers these practices for teams moving AI applications beyond prototypes.

0 comments Read more at DEV Community