Amazon Swaps Frontier LLM for 2B Model in Ticket Routing, Cutting Cost 98%

Amazon reportedly replaced Claude Sonnet 4.6 with a 2-billion-parameter model on one of its large internal ticketing systems, reporting similar quality at a fraction of the cost. Cost per ticket reportedly fell from about $0.0222 to roughly $0.0005. The gains came less from cheaper tokens than from changing the task: the smaller model outputs a single option index rather than generating a label and explanatory text. This also cut latency by about 78% and returned a confidence score that allows uncertain tickets to be flagged. Cost also depends on billing model, since a dedicated instance is cheaper at high volume but carries a fixed hourly floor.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in