System design interview guide: Building a durable distributed message queue

Designing a distributed message queue is a common system design interview challenge that tests multiple aspects of distributed systems. The core structural insight involves shifting from a transient in-memory queue to a partitioned, append-only log stored on disk, where each message has an offset. This approach enables durability, preserves order, and allows multiple consumers to read at their own pace. The interview typically lasts about 60 minutes, focusing on requirements, estimation, high-level design, and deep dives into trade-offs. The discussion often probes durability under failure, such as whether a message is lost if a broker crashes after accepting a write.