Model Cascade technique cuts LLM classification costs by routing easy tasks to cheaper models

A method called Model Cascade reduces the cost of large language model classification tasks by using a cheap 'proxy' model for most inputs and reserving an expensive 'oracle' model only for low-confidence cases. The system is calibrated offline using a sample of 500–1,000 records to determine a confidence threshold at which the proxy reliably matches the oracle's output. The approach is based on the BARGAIN framework, which offers statistical accuracy guarantees and reported up to 86% greater cost reduction than competing methods across eight datasets. A follow-up paper, Task Cascades, adds further optimizations—such as prompt rewriting and selective document retrieval—cutting costs an additional 48.5% on average. The BARGAIN library is available via pip and supports customizable proxy-oracle pairings across different AI providers.