Softmax
A function that turns a list of raw scores into probabilities that are all positive and add up to 1. It's how a model expresses how likely each option is.
Think of It Like This
It turns raw scores into a pie chart — every slice is positive and they all add up to 100%.
Softmax takes each score, exponentiates it, and divides by the running total so the results land between 0 and 1 and sum to 1. The exponent step exaggerates gaps, so a clearly leading score becomes a confident probability. It's the final step in classifiers and a key ingredient inside attention.
import numpy as np
def softmax(logits: np.ndarray) -> np.ndarray: """Turn raw scores into probabilities that sum to 1.""" shifted = logits - np.max(logits) # subtract the max for numerical stability exps = np.exp(shifted) return exps / exps.sum()