How One Team Scaled an Outbox Pattern to 3 Million Events Daily on Azure SQL
Engineers running a transactional outbox on Azure SQL Database Business Critical optimized a system processing roughly 3 million events per day and maintaining around 45 million rows in steady state. The core challenge was that the outbox table becomes a high-contention hotspot, absorbing continuous inserts, updates, and deletes while being polled by a never-idle scheduler. To enable horizontal scaling without table locks, the team used SQL Server's UPDLOCK, ROWLOCK, and READPAST hints, allowing concurrent worker instances to claim disjoint batches of up to 100 events safely. Keyset pagination replaced offset-based queries to keep read costs constant regardless of backlog depth, avoiding a performance collapse as queue depth grew. Publishing to Service Bus was deliberately kept outside the database transaction to prevent long-held connections and reduce overall resource consumption on a platform where compute directly drives cost.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in