Information Gain
A metric used in decision trees to measure the reduction in entropy or uncertainty about a target variable after splitting data on a specific feature.
Think of It Like This
Like playing Twenty Questions and asking a brilliant question that instantly eliminates half of the possible answers.
It guides the construction of tree-based models by selecting the feature that best separates the classes at each node. High information gain implies that the feature is highly predictive of the target variable. It is a foundational concept derived from information theory and Shannon entropy.