Why confident mistakes cost classifiers so heavily under cross-entropy loss

Cross-entropy is the loss function behind most classifiers, and it scores a model by the probability it assigned to the correct class, using the negative log of that probability. Plotted over class scores, the loss forms a landscape that is nearly flat where the model is right and climbs without limit where it is confidently wrong. The slope of that landscape gives the training update, so the largest corrections land exactly where the model errs most. The article illustrates this with a worked example in which a confident wrong prediction drops in loss over three gradient steps. It also frames the loss as a betting problem, where betting almost nothing on the true answer produces an ever-growing bill. A PyTorch example shows that nn.CrossEntropyLoss expects raw scores rather than probabilities.