Self-Hosting AI Models Rarely Beats Hosted Open-Weight APIs on Cost

When choosing between closed frontier APIs and self-hosted open-weight models, engineers often overlook a third option: hosted open-weight APIs that serve the same models without operational overhead. Closed frontier APIs cost roughly $2.50–$15 per million tokens, while hosted open-weight endpoints for models like Llama 4 run as low as $0.07–$0.90 per million tokens, and self-hosting on an H100 costs around $0.18 per million tokens — but only at 85% GPU utilization. In practice, production Kubernetes clusters average just 5% GPU utilization, which can inflate self-hosting costs up to 17 times compared to theoretical estimates. Additional operational work — version upgrades, monitoring, and capacity planning — adds $1,500–$4,000 per month in hidden costs not reflected in GPU bills. As a result, self-hosting only breaks even against hosted open-weight APIs at volumes exceeding 50 million tokens per day, a threshold most workloads never reach.