Reward Model
A secondary machine learning model trained explicitly to evaluate and score the outputs of a primary AI agent based on complex human preferences.
Think of It Like This
Like a strict food critic who tastes every dish a chef makes and assigns a numerical score based on presentation, taste, and creativity.
In RLHF, the reward model acts as a proxy for human judgment. It is trained on datasets of thousands of human-ranked comparisons (e.g., 'Response A is better than Response B'). Once trained, it provides the dense, automated reward signals necessary for algorithms like PPO to fine-tune the massive underlying language model.