TTFT vs Tokens Per Second: Why Optimising the Wrong Metric Wastes Effort

Two distinct metrics govern the latency of AI language model responses: Time to First Token (TTFT), which measures the delay before output begins, and Tokens Per Second (TPS), which measures generation speed once streaming starts. For interfaces where a human watches text arrive in real time, TTFT matters most, since readers absorb prose at roughly 5–6 tokens per second and most hosted models already exceed that threshold by a wide margin. Beyond that threshold, faster token generation offers no perceptible benefit to a reader, making every extra millisecond of TTFT the more meaningful bottleneck. In contrast, for agent pipelines or batch processes where no one watches intermediate output, TPS dominates total wait time and TTFT becomes negligible. Reporting a single blended latency figure obscures which lever actually needs pulling, often leading engineers to optimise the wrong variable and misread the results.