Gen AI

Min-P Sampling

Instead of adding up probabilities to hit a target, simply look at the most likely token, calculate a percentage of its probability, and mercilessly chop off anything smaller than that threshold.

Why Does This Exist?

Top-P (nucleus) sampling is the industry standard for catching the "long tail" of bad tokens, but it has a subtle mathematical flaw. Because Top-P relies on a running cumulative sum, it can be hijacked by a crowd of mediocre tokens.

Imagine the top token is "apple" at 40%. There are 60 other random, slightly-related words that each have a 1% probability. If Top-P is set to $P=0.90$ , it will keep "apple", and then it will sequentially absorb 50 of those mediocre 1% words to hit the 90% threshold. The model is now rolling a die where it has a massive 50% chance of drawing a highly questionable word, simply because the mediocre words mathematically "teamed up" to fill the quota.

Min-P was popularized by the open-source community (specifically llama.cpp) to solve this. Instead of a cumulative sum, Min-P uses a relative threshold based entirely on the top token.

Think of It Like This

A professor grading on a strict relative curve

Top-P grading is like a professor saying, "I will accept passing students into my advanced class until the class is 90% full." If the top student gets a 100%, and the next 40 students all get a 20%, the professor keeps letting the 20% students in until the room is mostly full. The standard of the room plummets.

Min-P grading is like a professor saying, "I don't care how many students I accept. You must score at least 10% as well as the top student." If the top student gets a 100%, the cutoff is a 10%. If the top student bombs and gets a 40%, the cutoff adjusts downwards to a 4%. You are judged solely on how you compare to the leader, not how you fill a quota.

How It Actually Works

The Relative Threshold

Min-P requires a user-defined threshold, usually between 0.01 and 0.1 (1% to 10%). The algorithm is incredibly fast and avoids the expensive sorting steps required by Top-K and Top-P:

Find the single highest probability in the distribution (the leader).
Multiply that probability by the Min-P setting to get the hard cutoff.
Throw away any token whose probability is less than that cutoff.
Re-normalize the survivors and sample.

Adapting to Confidence

Just like Top-P, Min-P adapts dynamically to the model's confidence, but it does so much more aggressively.

If the model is confident, the leader might be at 80%. If Min-P is set to 0.1, the cutoff is $80\% \times 0.1 = 8\%$ . Any token below 8% is instantly destroyed. The crowd of mediocre 1% tokens is wiped out in a single sweep, leaving only the leader (and maybe a strong runner-up).

If the model is unsure, the leader might only be at 10% (e.g., there are ten equally valid 10% synonyms). The cutoff becomes $10\% \times 0.1 = 1\%$ . Now, all the synonyms survive, preserving the model's creative options when there is no clear right answer.

Show Me the Code

Min-P is much simpler and computationally cheaper to implement than Top-P because it requires a single max() operation rather than a full sort() and cumsum().

1import numpy as np2
3def min_p_sampling_mask(probs: np.ndarray, min_p: float) -> np.ndarray:4    # 1. Find the probability of the leading token5    top_prob = np.max(probs)6    7    # 2. Calculate the dynamic cutoff threshold8    cutoff = top_prob * min_p9    10    # 3. Create a boolean mask of tokens to keep11    keep_mask = probs >= cutoff12    13    # 4. Zero out the rejected tokens14    safe_probs = probs.copy()15    safe_probs[~keep_mask] = 0.016    17    # 5. Re-normalize18    return safe_probs / np.sum(safe_probs)19
20# Example: The "Mediocre Crowd" scenario21# Top-P(0.9) would keep ALL of these 1% tokens to hit the sum.22# Let's see what Min-P(0.1) does.23probs = np.array([0.40] + [0.01] * 60) # 1 leader at 40%, 60 tokens at 1%24
25safe_probs = min_p_sampling_mask(probs, min_p=0.1)26
27print(f"Top Token Prob:  {np.max(safe_probs):.2f}")28print(f"Tokens Retained: {np.sum(safe_probs > 0)}")29
30# -> Top Token Prob:  1.0031# -> Tokens Retained: 1

With Min-P set to 0.1, the cutoff was 0.04 (40% * 0.1). Since the mediocre crowd was only at 1%, they were entirely annihilated. The model successfully recognized that 40% was a dominant lead and fell back to a deterministic, safe choice.

Watch Out For

Combining Min-P with Top-P

Because Min-P and Top-P are trying to solve the exact same problem (truncating the long tail dynamically), you should generally choose one or the other. Using both simultaneously makes the behavior of your generation pipeline highly unpredictable, as the two algorithms will fight over which one gets to aggressively truncate the distribution first.

The Quick Version

Min-P is a newer, highly effective alternative to Top-P for truncating the long tail of bad tokens.
It calculates a hard cutoff threshold by multiplying the single highest probability token by a user setting (e.g., 0.1).
It completely ignores cumulative sums, making it immune to being hijacked by a large crowd of mediocre, low-probability tokens.
It requires no sorting, making it computationally faster than Top-P or Top-K.
It adapts perfectly to model confidence: high confidence triggers a high, aggressive cutoff; low confidence triggers a low, inclusive cutoff.