Developer tests vLLM server setting, finds 68% cost reduction per token

·1 views

A developer conducted an experiment by renting an NVIDIA A100 GPU for under an hour to test a specific vLLM server flag called max_num_seqs. They compared setting the flag to 1 versus 8 while running the Qwen2.5-0.5B model with the same prompts and eight concurrent requests. Using an interleaved testing method to account for system variability, they found the setting of 8 reduced the cost per million output tokens by approximately 68%, dropping from about $0.75 to $0.23. The developer notes the baseline setting of 1 is unrealistically poor for production and results would vary with different models and workloads. They have released the raw data and an open-source tool called throttle-pro to help others measure such performance changes.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

TanStack npm supply-chain attack spreads via Dependabot update

On May 11, 2026, a worm compromised TanStack's release pipeline, publishing 84 malicious versions of 42 packages to npm with valid provenance. The attack leveraged a poisoned cache in a GitHub Actions workflow to steal publishing credentials. Hours later, a Dependabot update automatically pulled two compromised versions into an unrelated aviation-data project. When the maintainer merged the update, the worm used his credentials to publish 110 malicious versions of his own packages. The incident affected over 160 packages ecosystem-wide, including those from Mistral, before being contained.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

LangGraph recommended as primary Python framework for agentic AI in 2026

A 2026 developer analysis recommends LangGraph as the primary Python framework for agentic AI, citing its explicit graph model and large ecosystem. The Microsoft Agent Framework is advised for Azure-centric infrastructure as it merges AutoGen and Semantic Kernel. CrewAI is identified as the fastest option for prototyping multi-agent teams without production governance. The evaluation notes a migration deadline for Azure users, as Prompt Flow services will retire in April 2027.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Tutorial outlines four ffmpeg techniques for creating seamless video loops

The author details methods to eliminate visible cuts in looping videos using the open-source tool ffmpeg. Four techniques are presented, ranging from simple clip repetition to generating custom transition bridges. Each method addresses the problem where the last and first frames of a video do not match, causing a noticeable seam. The tutorial recommends normalizing video source files first to avoid issues with differing formats. These techniques can be performed directly with ffmpeg or through the author's browser-based tool, LoopVideo.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

High-Risk Vulnerabilities Found in 16 Popular Drupal Add-Ons

German cybersecurity agency CERT-BUND issued a high-risk advisory on September 23, 2026, covering 36 vulnerabilities in contributed Drupal projects. The batch of flaws, spanning identifiers from CVE-2026-96355 to CVE-2026-96398, includes risks like arbitrary code execution, privilege escalation, and data manipulation. Sixteen specific third-party Drupal modules, including Webform, Project Browser, and Commerce Decoupled Checkout, are named as affected. Drupal core itself is not impacted, but any installed version of the listed modules below the specified fixed version remains vulnerable. System administrators are advised to update the modules to their patched releases based on an internal inventory.

0 comments Read more at DEV Community