Log-Likelihood
The logarithm of the likelihood, which turns an unusable product of probabilities into a well-behaved sum.
A likelihood over independent observations is a product of probabilities, and that product is numerically hopeless: a thousand probabilities around 0.1 multiply to , which underflows to exactly zero in any float type. The objective becomes flat zero everywhere and there is nothing to optimise.
Taking the logarithm converts the product into a sum, , which is perfectly stable. This is safe rather than approximate: is strictly increasing, so it never moves the location of the maximum, only its value.
Negate it and you have the negative log-likelihood — your loss function, because optimisers minimise. Every standard loss is one of these: a categorical assumption gives cross-entropy, a Bernoulli gives binary cross-entropy, a Gaussian gives mean squared error. Note that log-likelihoods are unbounded below and only comparable on identical data, so the raw number is a relative measure, never an absolute score.