How Large Language Models Generate Text One Token at a Time

An LLM works by repeatedly predicting the next word, or token, given the text so far, then appending it and continuing the loop. A simple Python example shows how a model completes a function by scoring likely continuations and choosing the most probable one. Unlike basic phone autocomplete, accurate prediction requires understanding grammar, context, and code structure, which models learn from vast training data. Anthropic research on Claude suggests the model can plan ahead rather than working purely step by step, such as choosing a rhyme word before writing the line. The gradual appearance of answers in chat interfaces simply reflects this token-by-token generation process.