Unbounded queue kept a delivery text executor at one thread during a slow SMS morning
A delivery company's service sends customers a text with their two-hour delivery window using an asynchronous Spring executor configured with a maximum of fifty threads. When the SMS provider slowed sharply one October morning, the texts went out hours late, and drivers had already left cards at customers' doors. A thread dump revealed only one active thread, because the executor's default unbounded queue never filled, so the pool never grew beyond its core size of one. The team fixed it by raising the core size and restarting, which cleared the backlog in about ten minutes. They then moved notifications to a RabbitMQ queue with explicit consumer concurrency, bounded their remaining executors, and added monitoring with an alert on the age of the oldest queued task.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in