Bootstrap
A fundamental resampling method that involves repeatedly drawing random samples with replacement from a single dataset to estimate a statistical population.
Think of It Like This
Like drawing a card from a deck, writing down the suit, and putting it back before drawing again to figure out the overall composition of the deck.
Because it allows the same observation to appear multiple times in a newly generated sample, bootstrapping lets practitioners approximate the distribution of almost any complex statistic without collecting more data. It forms the backbone of ensemble methods like bagging, where each decision tree is trained on a different bootstrapped view of the original training set.