Skip to content
AI360Xpert
Visual Explainers
Visual explainer

Random Forests

Train hundreds of trees on random slices of data, let them vote, and watch individual errors cancel each other out — leaving a model that generalises far better than any single tree.

A single deep tree memorises every training point, tracing a jagged boundary that fails on new data.
A single deep tree memorises every training point, tracing a jagged boundary that fails on new data.

A single decision tree that is allowed to grow freely will carve the training set perfectly — and generalise poorly. Every quirk of the training data gets baked into the boundary. The solution is not a smarter tree; it is many trees, each kept slightly ignorant.

Grow Many Different Trees

A forest grows many trees, each trained on a different random bootstrap sample of the data.
A forest grows many trees, each trained on a different random bootstrap sample of the data.

Bootstrap sampling draws a new training set for each tree by sampling the original data with replacement. Each tree is also restricted to a random subset of features at every split, so no two trees make the same sequence of questions.

Every Tree Votes

Every tree casts a vote for its predicted class; the class with the most votes wins.
Every tree casts a vote for its predicted class; the class with the most votes wins.

At prediction time all trees are queried in parallel. For classification the majority class wins; for regression the predictions are averaged. No tree has authority — the decision is collective.

Averaging Shrinks the Error

Averaging many imperfect votes smooths out individual errors and lowers variance.
Averaging many imperfect votes smooths out individual errors and lowers variance.

Each tree errs in a different direction because each saw different data. When uncorrelated errors are averaged, positive and negative mistakes cancel and the variance of the combined prediction falls roughly as 1/N, where N is the tree count.

Where It Breaks

When all trees share the same noisy feature, their votes are correlated and the forest gains nothing.
When all trees share the same noisy feature, their votes are correlated and the forest gains nothing.

Feature randomness is the diversity engine. If the training data has one dominant but noisy feature, every tree will tend to select it at the root regardless of the random subset, making all trees nearly identical. Correlated errors do not cancel — the variance stays as high as a single tree, and the ensemble buys nothing.

The Quick Version

  • A single deep tree overfits by memorising the training boundary.
  • Bootstrap sampling gives each tree a different view of the data.
  • Random feature selection at each split forces further diversity.
  • All trees vote and the majority wins; errors in different directions cancel.
  • Correlated trees — sharing a dominant noisy feature — kill the variance benefit.

Related Concepts