Why Adding Commas to Numbers Improves LLM Arithmetic Accuracy

Large language models that use GPT-style tokenizers split digit sequences into chunks of up to three digits from left to right, meaning the same trailing digits in two different numbers often get split into completely different tokens. This misalignment causes the model to lose place-value consistency, making arithmetic on numbers longer than three digits unreliable. Formatting numbers with thousands separators — such as 1,234,567 — forces the tokenizer to break digits at power-of-1000 boundaries, creating stable, place-value-aligned tokens. Alternative tokenizer families like those used by Llama 2 and DeepSeek split every digit individually, which is more math-friendly but roughly triples token count on numeric text. Experts recommend normalizing number formatting at the prompt level as a mitigation measure, while routing any serious arithmetic to a dedicated tool.