Top-K
A decoding rule that limits a language model to its K most likely next words, then samples from just that shortlist. It trims off the long tail of unlikely words so the output stays coherent.
Think of It Like This
Order only from the menu's K bestsellers instead of the whole kitchen.
With top-k set to 40, the model ranks every possible next word by probability, keeps the best 40, and throws the rest away before choosing one. A small K keeps things safe and on-topic; a larger K lets in more variety. It pairs naturally with temperature, which decides how boldly the model picks from whatever shortlist top-k leaves behind.