Why Decision Trees Grow Instead of Learn, and Why That Instability Matters

Unlike logistic regression, decision trees are built by greedily selecting the best local data split at each step rather than through gradient descent optimization. Because finding a globally optimal tree is NP-hard, the algorithm commits to each split without backtracking, making the process fast but inherently approximate. This greedy structure also makes trees unstable: a small change in training data can alter the very first split and cascade into a completely different tree structure. In production settings, this means retraining a model monthly could silently shift the organization's stated explanation for outcomes even if prediction accuracy stays the same. Practitioners are advised to version and diff tree structures between retrains, not just track accuracy metrics.