SShortSingh.
Back to feed

SGLang cold start study shows GPU billing starts long before first user request

0
·1 views

A developer ran a performance test on October 5, 2026, using a T4 GPU on Google Colab with the SGLang 0.5.21 framework. They found that while the server was marked ready after approximately 387 seconds, the first user request wasn't completed until 563 seconds after billing began. This means 88% of the total billed time elapsed before any user request was served. The test highlighted a significant gap between a server being 'ready' and actually being capable of handling user requests efficiently. For comparison, a similar test using the vLLM framework showed a much shorter delay between readiness and request completion.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Regulators target 'dark pattern' exit costs as tech lock-in strategy

A tech industry commentator argues that the high cost of leaving a digital service, termed the 'exit layer,' is a deliberate business strategy rather than a technical side effect. This is exemplified by New York City's new 'Click to Cancel' law, which forces services to make cancellation as easy as sign-up. The same day, the free Twitter front-end Nitter appealed for survival funding, and a Yandex Cloud data center outage highlighted cloud dependency risks. The analysis states that exit costs are embedded in data portability, system integrations, team coordination, and institutional processes, making them difficult to quantify upfront.

0
ProgrammingDEV Community ·

UTF-8 vs GBK Encoding Differences Cause Script Failures Across Windows Machines

A common cause for scripts failing on servers after working on developer laptops is inconsistent text encoding defaults. On Windows, the default system encoding, or codepage, varies based on the OS locale, influencing functions like file operations and printing. For example, a developer's machine using UTF-8 and a production server using the Chinese GBK (cp936) encoding will handle text differently. This discrepancy can lead to corrupted file output, crashes when printing certain characters, and errors in file paths or subprocess calls. The core issue is that Python and other tools often implicitly use the system's locale-dependent encoding when none is explicitly specified.

0
ProgrammingDEV Community ·

Developer Successfully Runs Debian Linux on Samsung Galaxy Book Go Laptop

A developer has published a guide detailing how to run the Debian sid Linux distribution on a Samsung Galaxy Book Go, a laptop with a Snapdragon 7c Gen 2 ARM processor designed for Windows. The process required a custom device tree patch and shell scripts to enable functionality for key hardware like the GPU, Wi-Fi, keyboard, and battery. This work was necessary because the laptop lacks upstream Linux support and Samsung provides no driver pack, forcing reliance on a draft community device tree originally made for a different device with the same processor. The developer tested configurations through multiple reboots, logging 26 boots over two days to correct incorrect hardware mappings. The article itself was written and verified using the Debian system running on the Galaxy Book Go.

0
ProgrammingHacker News ·

GitHub project 'Instant Factory' generates software projects from configuration

A GitHub project called 'Instant Factory' by developer Mikola Lysenko was shared on Hacker News. The tool automatically generates complete software projects from declarative configuration files. It aims to reduce initial setup time by creating boilerplate code, build systems, and documentation. The project received moderate community engagement with six points and three comments on the platform.