Skip to content
AI360Xpert
Glossary
Definition

Data Drift

A change in the inputs a model receives after it was trained, measured by comparing production data against the training window.

The useful thing about input drift is that you can detect it without labels, which is why it's what most monitoring measures. Compare each feature's distribution against a reference — and the reference should be your training data, not last week — using a population stability index, a Kolmogorov-Smirnov test, or a chi-squared test over categorical bins.

The best single instrument is a domain classifier: train a model to tell a training row from a production row. If it can't do better than chance, nothing moved. If it scores 0.85, its feature importances name exactly which columns did.

Two disciplines make it worth having. Set thresholds in advance, per feature, and correct for the number of tests — 200 features tested nightly at p<0.05p < 0.05 produces ten false alarms a day, which teaches everyone to ignore the channel.