How AdaBoost Turns Weak Decision Stumps Into a Powerful Classifier

AdaBoost is a machine learning boosting algorithm that combines hundreds of simple, near-random classifiers called decision stumps to build a highly accurate ensemble model. Each stump makes just one binary split on one feature, performing only marginally better than random guessing on its own. The algorithm assigns a weight to every training point, increasing the weight of misclassified examples after each round so subsequent stumps focus on the hardest cases. Each stump's contribution to the final vote is scaled by a confidence value called alpha, calculated from its weighted error, ensuring accurate stumps dominate and poor ones are discounted or flipped. This iterative reweighting process is mathematically equivalent to gradient descent on an exponential loss function, which guarantees that training error decreases with each added stump.