Core ML

Counterfactual Explanations

Instead of explaining why a model rejected you, a counterfactual explanation tells you the exact minimum changes you need to make to your profile to get approved.

A counterfactual explanation tells the user the exact minimum change required to flip a model's negative prediction into a positive one.

Why Does This Exist?

Techniques like shap and lime are fantastic for data scientists. If a model denies a user a loan, SHAP can output a beautiful waterfall chart showing that Debt pushed the prediction down by 15%, while Age pushed it up by 5%.

But imagine showing that SHAP chart to the user whose loan was just denied. The user doesn't care about marginal feature contributions or game theory. They care about one thing: "What do I need to do to get approved next time?"

SHAP cannot answer that question. It only explains the past. Counterfactual explanations exist to explain the future. They provide actionable recourse. A counterfactual algorithm searches the mathematical space around the user's data and finds the absolute closest point where the model's prediction flips from "Denied" to "Approved." It translates a mathematical rejection into human advice: "If you increase your income by $5,000 and pay off$ 2,000 in credit card debt, you will be approved."

Think of It Like This

Think of model explainability like getting a failing grade on a math test.

SHAP is like the teacher handing back the test with red marks. It explains exactly why you failed (e.g., "You lost 15 points on algebra and 10 points on geometry"). It is a perfect accounting of the past.

Counterfactuals are like the teacher giving you a study plan. It tells you exactly what to do next (e.g., "If you study algebra for two more hours a week, you will pass the next test"). It is a roadmap for the future.

How It Actually Works

Generating a counterfactual explanation is an optimization problem. You are trying to find a new, fake data point (the counterfactual) that satisfies three strict rules.

1. It Must Cross the Boundary

If the current prediction is 0 (Denied), the counterfactual data point must produce a prediction of 1 (Approved) when fed into the original black-box model.

2. It Must Be Minimal

You could easily get John Doe approved for a loan by artificially changing his income to $5 million. But that is useless advice. The algorithm uses a distance metric (like Euclidean or Manhattan distance) to find the counterfactual point that is mathematically closest to John Doe's original data. It seeks the path of least resistance.

3. It Must Be Actionable (Feasibility)

This is the hardest constraint. Algorithms don't understand reality. If a purely mathematical algorithm tries to find the shortest path to approval, it might suggest: "You will be approved if you decrease your Age from 45 to 30."

This is mathematically correct but physically impossible. To generate valid counterfactuals, you must lock immutable features (like Age, Race, or Past Defaults) so the algorithm cannot change them. You can also enforce directionality (e.g., "Education Level can only go up, not down").

Show Me the Code

Generating counterfactuals is often done using libraries like DiCE (Diverse Counterfactual Explanations) developed by Microsoft. Here is a conceptual example of how you constrain the algorithm to produce actionable advice.

1import dice_ml2
3def generate_actionable_recourse(4    model, 5    training_data: pd.DataFrame, 6    user_data: pd.DataFrame7):8    """Generates counterfactual advice for a rejected user."""9    10    # 1. Initialize the DiCE data structure11    d = dice_ml.Data(12        dataframe=training_data,13        continuous_features=['income', 'debt', 'age'],14        outcome_name='approved'15    )16    17    # 2. Wrap the black-box model18    m = dice_ml.Model(model=model, backend='sklearn')19    20    # 3. Initialize the Explainer21    exp = dice_ml.Dice(d, m)22    23    # 4. Generate the counterfactual24    # We strictly forbid the algorithm from changing 'age' (immutable)25    # and require it to output a positive prediction (desired_class=1)26    dice_exp = exp.generate_counterfactuals(27        user_data, 28        total_CFs=1, 29        desired_class=1, 30        features_to_vary=['income', 'debt'] # Only allow these to change31    )32    33    return dice_exp.visualize_as_dataframe()34
35# Output might look like:36# Original: Income 45k, Debt 20k -> Denied37# Counterfactual: Income 55k, Debt 15k -> Approved

Watch Out For

The Moving Goalpost

Counterfactuals assume the model itself will not change. If you tell a user, "Increase your income by $5k to get approved," and they spend a year doing exactly that, they expect to be approved. But if your data science team retrains the model six months later with new economic data, the decision boundary will shift. The user might reapply with the$ 5k increase and get denied again. This creates massive trust and liability issues. If you offer a counterfactual, you are often implicitly making a promise.

The Quick Version

Standard explainers (like LIME and SHAP) explain why a model made a decision in the past.
Counterfactuals explain what a user needs to change to get a different decision in the future.
The algorithm searches for the smallest possible change to the user's data that successfully flips the model's prediction.
To be useful, counterfactuals must be constrained. They cannot recommend impossible actions, like decreasing someone's age or erasing a past bankruptcy.
Counterfactuals provide the ultimate "actionable recourse," turning opaque AI rejections into step-by-step improvement plans for the user.

Related concepts

SHAP Values