Boosting
A powerful sequential ensemble technique that trains a series of weak models where each new model specifically focuses on correcting the errors of the last.
Think of It Like This
Like a relay race where every new runner specifically practices the exact hurdles that the previous runner tripped over.
Unlike bagging, which builds models in parallel to reduce variance, boosting builds them sequentially to reduce bias. The algorithm assigns higher weights to data points that previous iterations misclassified, forcing the next tree to pay closer attention to hard examples. Algorithms like XGBoost and Gradient Boosting are standard choices for tabular data.