Edge-based dynamic pricing cuts p99 latency using WebAssembly and quantized ML models

·1 views

A global retailer saw its conversion rate drop 22% within ten minutes during Black Friday, traced to a p99 latency spike reaching 2.4 seconds. The bottleneck was a centralized Python pricing microservice struggling with real-time purchase-propensity and price-elasticity inference under heavy request volume. The article argues that moving pricing logic to the network edge avoids round trips to regional data centers and reduces inference time. It proposes running ONNX models compiled to WebAssembly on edge runtimes, with models quantized from FP32 to INT8 to shrink their size by up to 75% at a marginal accuracy cost. The approach also separates offline training from online edge execution and incorporates Zero Trust authentication.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

No Firewall Exists for Prompt Injection in AI Agents, Engineer Argues

An applied AI engineer at LuumenAI, a platform for SAP and ERP observability, argues that language models cannot reliably separate instructions from untrusted input, so conventional security controls fall short. The piece notes that the model treats operator prompts, user requests, and pasted content as one stream, which means text an attacker places in front of it carries real authority. It cites the EchoLeak vulnerability in Microsoft 365 Copilot, disclosed by Aim Labs in June 2025 and rated CVSS 9.3, in which a crafted email caused the assistant to exfiltrate sensitive data without any user action. The author frames the fix as controlling what the model can read and what it is permitted to do, and says a follow-up paper will address containment after the model acts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Slothshot Uses a Local Gemma Vision Model to Rename Screenshots by Content

A developer built slothshot, a Go command-line tool that renames screenshots after what they show so they can be found through search. The author had 39 screenshots across three default naming schemes from macOS, GNOME and Flameshot, none of which described their contents. The tool runs Gemma 4 E4B, a 4-bit open-weight vision model served locally through llama.cpp, so no images leave the machine. It offers a dry-run default, an apply mode, an undo option that restores old names from a log, and a watch mode that names new screenshots as they appear. The author acknowledges errors, such as misspelling a brand name and omitting a key error message, but argues that imperfect names are still easier to search than default timestamps.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Blocking Embedding Calls Caused a 40% Request Drop in a vLLM Semantic Router Under Load

A FastAPI gateway wrapping the vLLM semantic router began silently discarding requests during a 2 AM traffic spike, with queue latency rising from about 80ms to 4.8 seconds. The root cause was a synchronous embedding computation inside the router call, which blocked the asyncio event loop and prevented it from accepting new requests. Requests that exceeded the internal five-second timeout were discarded without any error response. The fix moved the router call into a thread pool using asyncio.to_thread so the event loop stays free, and the author also benchmarks routing latency against a multi-provider relay.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Solo developer's one-bullet puzzle game Bloop changed after player feedback

Bloop is a mobile puzzle game built entirely by one developer, in which each level gives the player a single bullet to bounce off walls and clear every target. The game is now live on Google Play and the App Store with 300 levels, though the developer describes it as still in progress. Early feedback led to a switch from touch-based aiming to a slingshot-style pull-back control, and animated, reactive enemies that proved the most popular change. Some levels were found to be unbeatable, so the developer built an automated checker that tests every level with the game's physics and requires a realistic margin for aiming or timing. All 300 levels passed the checker on October 15, and future plans include new enemy types and additional galaxies.

0 comments Read more at DEV Community