Core ML

Testing ML Systems

You can't write a unit test that says 'assert model.predict(image) == dog', because the model is probabilistic. Testing ML systems requires testing the data going in, checking specific behavioral invariants, and tracking aggregate metrics on golden sets.

Traditional software testing asserts exact outputs. ML testing relies on Data Tests (inputs), Behavioral Invariants (logic bounds), and Golden Sets (aggregate accuracy).

Why Does This Exist?

In traditional software, you write unit tests: assert add(2, 2) == 4. If the test passes, the code works.

In Machine Learning, this doesn't work. Models are probabilistic black boxes. If you train a new model and it predicts 3.99 instead of 4, is it broken? What if it predicts 4.1? You cannot hardcode exact output assertions for a neural network.

Furthermore, an ML model can fail silently. The code compiles, the API returns a 200 OK, but because the input data distribution shifted, the model is confidently outputting garbage predictions.

Testing ML Systems requires a completely different paradigm. We have to test the inputs (Data Tests), test the logical boundaries (Behavioral Tests), and test the aggregate performance (Golden Sets).

Think of It Like This

Testing traditional software is like testing a calculator. You punch in $5 \times 5$ , and if it doesn't say $25$ , it's broken.

Testing an ML system is like testing a human student. You can't guarantee they will answer every single question on a history test identically every time you ask them. Instead, you check if they studied the right book (Data Tests). You make sure they don't say the Civil War happened in 1995 (Behavioral Bounds). And you give them a standardized test where you expect them to score at least an 85% (Golden Set).

How It Actually Works

A robust ML testing suite operates at three distinct layers:

1. Data Tests (Pre-Train & Pre-Inference)

If garbage goes in, garbage comes out. Data tests run before the model ever sees the data. Tools like Great Expectations allow you to write unit tests for your dataframes:

expect_column_values_to_not_be_null("age")
expect_column_values_to_be_between("income", 0, 1000000) If the input data violates these rules, the ML pipeline halts immediately.

2. Behavioral Tests (Post-Train)

Also known as Checklist Testing. We cannot test exact outputs, but we can test directional and invariant logic.

Invariance Tests: If I change a feature that shouldn't matter, the prediction should not change. (e.g., Changing the applicant's gender should not change the credit_score prediction).
Directional Expectation Tests: If I change a feature in a specific direction, the prediction should move in a specific direction. (e.g., If I increase the number_of_bathrooms, the house_price prediction should strictly increase or stay flat, never decrease).

3. Golden Set Tests (Pre-Deployment)

You keep a highly curated, static dataset of edge cases and critical examples called a Golden Set. Before any model is promoted to production, it must run inference on the Golden Set. You assert that aggregate metrics (like Recall or F1) do not drop below a strict threshold (e.g., assert new_model_f1 >= 0.85), and that it doesn't regress on critical business examples.

Show Me the Code

Here is how you might write a Behavioral Invariance test in Python using pytest.

1import pytest2import pandas as pd3
4def test_gender_invariance(trained_model):5    # 1. Create a baseline applicant6    base_applicant = pd.DataFrame([{7        "income": 85000,8        "credit_history": 5,9        "gender": "Male"10    }])11    12    # 2. Create the exact same applicant, but change the gender13    mutated_applicant = base_applicant.copy()14    mutated_applicant["gender"] = "Female"15    16    # 3. Get predictions17    base_pred = trained_model.predict_proba(base_applicant)[0][1]18    mutated_pred = trained_model.predict_proba(mutated_applicant)[0][1]19    20    # 4. Assert Invariance21    # The probability of loan approval should be mathematically identical22    # If the model learned a gender bias, this test will fail the CI/CD pipeline!23    assert abs(base_pred - mutated_pred) < 1e-5, \24        "Model violates gender invariance!"

Watch Out For

Data Leakage into the Golden Set. Your Golden Set is the ultimate gatekeeper for deployment. If your training pipeline accidentally includes data from the Golden Set, your model will perfectly memorize those examples. Your aggregate tests will pass with flying colors, but the model will immediately fail when exposed to real-world, unseen data. The Golden Set must be vaulted and completely inaccessible to the training script.

The Quick Version

You cannot write exact assert output == X unit tests for probabilistic ML models.
Data Tests ensure the inputs to the model are clean, bounded, and correctly typed.
Behavioral Tests assert logical boundaries, such as Invariance (changing an irrelevant feature shouldn't change the output) and Directional Expectations.
Golden Sets are static, highly curated datasets used to assert that aggregate metrics (like Accuracy or F1) haven't regressed before deployment.