Self-hosting an LLM beats frontier APIs at about 16M tokens a day, but needs far more traffic to beat cheap open-model APIs

Running an open-weight model on one NVIDIA H100 costs less than a frontier API such as Claude Sonnet 5.5 once usage reaches roughly 16 million tokens daily. Beating the low-cost per-token APIs serving the same open models requires about 240 to 400 million tokens per day. The analysis converts each API price into a blended rate using a 3:1 input-to-output token mix, with the H100 priced at $2.60 per hour. Because frontier models cost 15 to 26 times more per token than open models on per-token APIs, nearly any busy GPU beats them, while the cheaper APIs are harder to undercut. The comparison also notes that swapping a frontier model for an open one changes the model itself and only saves money if the open model passes the user's evaluations, and that idle GPU hours still incur cost, so utilization largely determines the outcome.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in