In-Context Learning Explained: How LLMs Adapt From Examples Without Updating Weights

Large language models can perform new tasks using only a few examples provided within a prompt, a capability known as in-context learning. Unlike traditional training, this process does not alter the model's internal weights; instead, the model uses the examples as a template to predict the most likely output. Researchers attribute this behavior to mechanisms such as induction heads, which detect and apply repeated patterns within the prompt. However, in-context learning has notable limitations, including restricted context length, difficulty with complex tasks, and a risk of overfitting to the given examples. Experts note that while the capability resembles generalization, it remains a constrained form of pattern completion rather than the deeper rule-learning seen in human cognition or fine-tuning.