Engineer Debugs vLLM KV Cache Corruption That Caused Morning Outage

·6 views

An on-call engineer was alerted at 8am to a production outage caused by KV cache corruption in a vLLM PagedAttention model-serving setup, with peak traffic hitting 14,720 requests per second. The system was throwing cache corruption exceptions when querying the KV store through the PagedAttention layer, bringing the service down entirely. Three initial fixes — adjusting NCCL timeouts, restarting Kubernetes pods, and checking GPU health — all failed, consuming roughly 45 minutes of debugging time. The root cause turned out to be a single-line error in the Dockerfile. Following the incident, the team added Grafana and PagerDuty alerts to catch NCCL communication failures and high all-reduce latency before users are impacted in the future.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Browser Use integrates with Scrapeless to run AI agents in anti-detection cloud browsers

Browser Use, a framework that lets large language models control a real browser, can now be connected to Scrapeless's cloud-based Scraping Browser via a single WebSocket endpoint using the Chrome DevTools Protocol. The integration addresses a core problem with local browser automation: headless Chromium exposes telltale signals such as the navigator.webdriver property, default headless fingerprints, and a traceable IP address that anti-bot systems routinely flag. By pointing the cdp_url parameter to Scrapeless, developers offload the browser runtime, fingerprint management, and residential IP routing to the cloud without changing any agent logic. The setup requires only a Scrapeless API key and supports session-level proxy country selection across more than 195 regions. New Scrapeless accounts receive free Scraping Browser runtime to get started at no initial cost.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

xAI and Cursor Set to Launch Joint AI Coding Model, Delayed for Efficiency Gains

xAI and Cursor, now operating under SpaceX's umbrella following a $60 billion all-stock acquisition of Cursor's parent company Anysphere in June, are preparing to release their first jointly developed AI model as early as Wednesday. The launch was pushed back by roughly two days — not due to capability issues, but to improve the model's processing efficiency and cost-effectiveness. According to a staff memo reported by The Information, the model is designed to handle information quickly and is said to be competitive with Anthropic's Opus 4.8 and OpenAI's GPT 5.5 in certain respects, though no benchmarks or pricing details have been released. The collaboration addresses a key constraint Cursor previously faced: limited access to computing power, which had been identified as a ceiling on its growth as an AI coding tool. With SpaceX's compute resources now backing the product, the joint model is expected to make Cursor a more vertically integrated and cost-competitive offering in the enterprise AI coding market.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Swiss compliance site releases two open datasets on wages and SME deadlines

A developer building Canton Compliance Hub, a free multilingual resource for small businesses in Switzerland, has published two structured datasets under the CC BY 4.0 license. The first dataset covers minimum wage rules across all 26 Swiss cantons and three cities, reflecting the country's lack of a national pay floor and sourcing figures from cantonal, municipal, and federal pages. The second dataset outlines 12 recurring federal compliance deadlines that Swiss SMEs must meet annually, including VAT returns, AHV declarations, and withholding tax obligations. Each row in both datasets includes a verified source URL, ensuring figures can be independently checked without relying on the publisher. The datasets are available in both CSV and JSON formats, with the JSON version serving as the multilingual source of truth from which the flat CSV is automatically generated.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

DIY Builder Constructs 8,192-Core GPU Using 13-Cent RISC-V Microcontrollers

A hardware enthusiast known as Bitluni has built a custom GPU by connecting 8,192 WCH CH570 RISC-V microcontrollers, each costing about $0.13, into a single parallel computing array. The project originated from a bulk chip purchase left over from earlier cluster experiments. Each CH570 chip runs at up to 100 MHz with 12 KB of SRAM, with groups of 32 reporting to larger controller chips to simplify communication. The build required six-layer PCBs, individual crystals per chip after a shared clock scheme failed, modular blade boards connected via PCIe edge connectors, and a 3 kW power supply delivering over 650 A on a 3.3 V rail. Firmware was flashed automatically using a pogo-pin fixture mounted on a printer toolhead guided by a Python script.

0 comments Read more at DEV Community