Logarithm

$\log_b x$ is the exponent that turns $b$ into $x$: since $2^3 = 8$, $\log_2 8 = 3$. In machine learning $\log$ with no base written means the natural log, base $e$, and that convention is worth stating out loud because base 2 appears constantly in information theory. Base 2 gives bits, base $e$ gives nats, and the conversion factor is $\ln 2 \approx 0.693$.

Three properties are why it appears in nearly every loss function. It turns products into sums, $\log(ab) = \log a + \log b$, which converts a product of thousands of per-example probabilities into a sum that a computer can actually represent. It is monotonically increasing, so whatever maximises a likelihood also maximises its log — the transformation is free. And it stretches the region near zero, so the difference between a predicted probability of 0.01 and 0.001 becomes a difference of 2.3 rather than 0.009, which is what gives a confidently wrong prediction a large gradient.

The catch is $\log 0 = -\infty$. Any probability that underflows to exactly zero produces an infinite loss or a nan, which is why stable implementations work in log space from the start rather than exponentiating and taking a log at the end.