Measuring a Server: Latency Knees, Misleading Percentiles, and Little's Law

The author built a deliberately simple single-worker server to study how latency behaves under load. Ramping request rates revealed a 'knee' where tail latency rises sharply well before the server is fully utilised. At about 70% busy, the 99th percentile latency had already more than doubled. The author also argues that averages hide slow outliers, so percentiles better reflect what users actually experience. A further experiment showed that fan-out amplifies this effect, since a page is only as fast as its slowest dependency.