VIDRAFT's VKAE Engine Hits 23x GPU Speedup and 10K Tokens/sec on Nvidia B200

·1 views

South Korean Pre-AGI startup VIDRAFT has unveiled VKAE, a proprietary inference acceleration engine for large language models. Benchmarked on a single Nvidia B200 GPU, the system achieved roughly 10,000 tokens per second on the Qwen3.5-35B-A3B model, compared to a baseline of approximately 455 tokens per second. This represents up to a 23-fold improvement in GPU utilization without any reported degradation in output quality. VKAE exposes an OpenAI-compatible API, allowing it to integrate seamlessly into existing LLM deployment pipelines. The engine is positioned as competitive with established open-source frameworks like vLLM and TensorRT-LLM, as well as commercial providers such as Groq and Cerebras.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer builds free no-signup GPS altitude finder as a Progressive Web App

A solo developer has launched Zirvə, a free Progressive Web App that displays a user's elevation above sea level without requiring an account, installation, or ads. The app primarily uses the Open-Meteo terrain elevation API for accuracy within 1–3 meters, falling back to the device's raw GPS altitude reading when offline. Because GPS chips measure vertical position far less reliably than horizontal position, the app clearly labels GPS-sourced readings as approximate to avoid misleading users. A service worker enables the app to load from cache when there is no internet connection, making it usable in remote areas such as above the treeline. The developer plans to add an elevation profile for tracked sessions and explore barometric pressure as an additional offline fallback.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Python's subprocess.terminate() Often Fails to Kill Child Processes

When a deployment script exits cleanly, child processes it spawned can silently keep running, holding ports or file locks and breaking subsequent deploys. Python's process.terminate() sends a SIGTERM signal, which is merely a request that the target process can ignore, defer, or handle in unexpected ways. The process.kill() method sends SIGKILL, which the kernel enforces unconditionally, but it must be used as a fallback after SIGTERM times out. A further complication arises when shell=True is used or when child processes spawn their own subprocesses, since the signal reaches only the direct child and not the entire process tree. A reliable termination pattern requires sending SIGTERM first, waiting with a timeout, escalating to SIGKILL if needed, and accounting for grandchild processes to ensure nothing is left running.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Is Reshaping Software Engineering — But Human Judgment Still Matters

Since ChatGPT launched in November 2022, AI tools have dramatically lowered the barrier to building software, enabling non-programmers to create custom applications and automate tasks that once required professional developers. Large language models have made prototyping faster and cheaper, allowing individuals and small businesses to validate ideas without deep technical knowledge. However, one software engineer argues that understanding code fundamentals remains essential, as human judgment is needed to evaluate AI-generated code for security, scalability, and performance. AI has also transformed how developers study, offering more interactive and personalized learning experiences than traditional methods. The author cautions that while AI will keep improving, engineers should filter out the constant hype and focus on building durable skills rather than chasing every new tool or model.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Test TypeScript AI Agents Without Making Real Model API Calls

Developers building AI agents in TypeScript often skip proper testing, relying on live runs that can miss critical failures like runaway loops or malformed tool responses. A lightweight interface called ModelClient can be used to inject a scripted fake client during tests, replacing real API calls entirely. The scripted client replays predefined responses in sequence, records all outgoing requests, and clamps to its last step to simulate infinite-loop scenarios safely. This approach allows developers to assert on turn limits, budget checks, and tool result pairing before any broken request reaches the API. The technique keeps production code unchanged while making edge-case agent behaviour fully testable in isolation.

0 comments Read more at DEV Community