Normal Distribution

Also called the Gaussian, and written $\mathcal{N}(\mu, \sigma^2)$. Two numbers fix it completely: where it is centred and how wide it is. Roughly 68% of its mass sits within one standard deviation of the mean and 95% within two.

It appears everywhere for a real reason rather than a conventional one — the central limit theorem says that sums of many independent contributions tend toward it regardless of what they individually looked like. Measurement noise is usually a sum of many small effects, so it is often approximately normal.

Its most load-bearing appearance in machine learning is hidden. Assuming normally distributed errors and taking the negative log-likelihood gives you mean squared error — the square in MSE comes directly from the square in the Gaussian's exponent. So every time you reach for MSE you are assuming symmetric, thin-tailed errors, which is exactly why MSE handles outliers badly. It is also the distribution weight initialisation schemes sample from, and the prior that L2 regularisation encodes.