TF-IDF

Term frequency times inverse document frequency: $\text{tf} \times \log(N / (1 + \text{df}))$, where $\text{tf}$ counts a token in one document, $N$ is the number of documents, and $\text{df}$ counts how many contain the token at all.

Read it as a rarity discount. A token frequent here and rare elsewhere scores highest. A token in every document has $\text{df} \approx N$, so the fraction lands near 1 and the log near 0 — grammar cancels itself out, which is why stopword removal matters much less once you're using TF-IDF.

Two practical notes. L2-normalise the rows or long documents dominate every distance, and once you do the dot product between two rows is their cosine similarity. And df is counted over whatever corpus you fit on, so fitting on all your documents leaks test information into every training weight.