In-Context Learning Lets Language Models Adapt to Tasks Without Retraining

In-context learning teaches a language model a new task by placing examples directly in the input prompt, leaving its internal parameters unchanged. Unlike traditional machine learning, which typically requires retraining or fine-tuning with new data, this approach avoids the time and cost of updating the model. It works because large language models such as GPT-3 and Llama 2 are trained on diverse text and learn to recognize and imitate patterns shown to them. The method is practical for tasks that change often or lack sufficient training data, and it requires only clear instructions and a few examples. The article illustrates this with a few-shot prompt that pairs countries with their capitals and asks the model to complete the pattern for a new country.