Blocking Embedding Calls Caused a 40% Request Drop in a vLLM Semantic Router Under Load

A FastAPI gateway wrapping the vLLM semantic router began silently discarding requests during a 2 AM traffic spike, with queue latency rising from about 80ms to 4.8 seconds. The root cause was a synchronous embedding computation inside the router call, which blocked the asyncio event loop and prevented it from accepting new requests. Requests that exceeded the internal five-second timeout were discarded without any error response. The fix moved the router call into a thread pool using asyncio.to_thread so the event loop stays free, and the author also benchmarks routing latency against a multi-provider relay.