Input Tokens Dominate LLM Token Volume, But That Is Not the Same as Wasted Spend
A telemetry analysis over a five-day window found that 99.13% of recorded model tokens were input tokens, suggesting surrounding context can far outweigh the model's responses. The author cautions that input tokens are not automatically wasteful and that token share does not equate to share of billing cost. Costs should be estimated by applying provider rates to each token category, and cost remains unmeasured where rates or cache treatment are unknown. The findings do not show that all input was overhead or that a specific reduction would preserve output quality. Verifying those points would require separate, controlled measurements.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in