How Least-Connections Load Balancing Outperforms Round-Robin Under Traffic Spikes

A developer at a tech platform recounts how a sudden API traffic spike exposed the limitations of their round-robin load balancer, which distributed requests evenly without accounting for each backend's actual workload. The core problem was that one node would become overloaded while others remained idle, causing 502 errors for users. The solution was switching to a least-connections algorithm, which routes each incoming request to the backend currently handling the fewest active connections, enabling fairer distribution during bursts. A slow-start window was also added to prevent new servers from being overwhelmed immediately upon joining the pool. The author shares a minimal Go implementation contrasting the naive round-robin approach with the improved least-connections design to illustrate the practical difference.