Three-Day Bug Hunt Reveals Token Mismatch, Not Model Weakness, Behind Classification Failures

A developer's ticket-classification service began returning incorrect fallback labels for long, non-English messages after a switch to a cheaper inference endpoint. Initial suspicion fell on the new model's reasoning ability, but controlled testing revealed that 19 of 50 tickets failed only beyond a specific input length threshold. The root cause was a mismatch between the client's character-based prompt truncation and the new endpoint's tokenizer, which split German and Japanese text into roughly twice as many tokens per character as English — silently cutting the classification instruction before the model could see it. The fix required replacing the character-based length guard with a token-based one aligned to the endpoint's tokenizer, adding a sentinel marker to detect silent truncation, and writing regression tests with long non-English inputs. No changes to the model itself were necessary, highlighting how infrastructure assumptions rather than model quality can cause subtle, consistent AI service failures.