Gen AI

Post-Training Evaluation

You just spent $5,000 fine-tuning a model to answer customer support tickets. How do you know it actually works? Post-training evaluation requires testing it against a prompted baseline to prove the tune was worth it, and checking for regression to ensure it didn't forget how to speak English.

Why Does This Exist?

It is incredibly easy to make a model worse through fine-tuning. If you train a model on 10,000 examples of support tickets, the training loss graph will trend downward beautifully, making you think the model is learning. But training loss only measures how well the model memorized the training data.

When you deploy it, you might find that it hallucinates company policy, refuses to answer basic greetings, or performs worse than the raw base model did with a good prompt. Post-training evaluation is the rigid, mandatory testing framework that answers two questions: "Did the fine-tune actually improve the target task?" and "Did it break anything else in the process?"

Think of It Like This

The new chef's trial run

You hire a chef who makes decent food across the board (the base model). You send them to a 3-week intensive seminar exclusively on making soufflés (fine-tuning).

When they return, you don't just ask them if they feel confident (training loss). You make them bake a soufflé and compare it to the one they made before the seminar (the prompted baseline).

Crucially, you also ask them to make a simple omelet. If the soufflé is perfect but they have completely forgotten how to scramble an egg, the training was a failure (regression).

How It Actually Works

A proper post-training evaluation requires two distinct testing suites.

1. The Task-Specific Test (Did the tune help?)

You must evaluate the fine-tuned model on a held-out dataset of the target task—data it never saw during training. However, the most common mistake is comparing the fine-tuned model against the raw base model. That is a meaningless comparison. You must compare the fine-tuned model against the base model equipped with a highly optimized, few-shot prompt.

If the base model with 5 examples in the prompt scores 85% accuracy, and your expensive fine-tuned model scores 86% accuracy, the fine-tune was a failure. The cost of maintaining a training pipeline is not worth a 1% gain over prompt engineering. You must prove the fine-tune significantly beat the prompted baseline.

2. The Regression Test (Did it break anything?)

Because of Catastrophic Forgetting, pulling a model's weights toward a narrow task often destroys its general capabilities. You must run the fine-tuned model through a suite of general benchmarks (like MMLU for knowledge, or HumanEval for coding) and compare those scores to the base model.

If your medical fine-tune improved medical accuracy by 10%, but dropped general reasoning scores by 30%, the model has become brittle and will likely fail in production when users ask unexpected questions. This is why replay data is used during training to anchor the model.

Watch Out For

Data Contamination

If a single example from your evaluation dataset accidentally slipped into your fine-tuning dataset, the evaluation is entirely void. The model will score 100% on that example because it memorized the answer, falsely inflating the perceived quality of the fine-tune. Strict hashing and deduplication between the train and test splits are mandatory.

The Quick Version

Training loss going down does not mean the fine-tune was successful; it often just means the model memorized the format.
Task Evaluation: You must prove the fine-tune significantly outperforms the base model using a highly optimized, few-shot prompt.
Regression Testing: You must run general benchmarks to ensure the model didn't suffer catastrophic forgetting and lose its foundational reasoning.
Never evaluate on data the model saw during training.

Related concepts

Quantization Quality Tradeoffs