Counterfactual Explanations

A job model estimates a 75% interview probability. “Two more years of experience” changes the prediction, but it is a miserable recommendation for someone applying this week. “Fifty hours of interview practice” may be slower numerically and more useful in reality.

A counterfactual describes the smallest meaningful change that reaches a desired outcome.

Search for an Actionable Counterfactual

Adjust work experience and interview preparation. The target is a 90% interview probability, but the two changes have very different costs.

Years of experience2 yearsInterview preparation10 hours

Interview probability

69%

Target

90%

Time cost

104.25 weeks

Status

keep searching

A counterfactual should be feasible and actionable. The shortest numeric path can still be useless to the person receiving it.

Trade years of experience against interview preparation and search for a feasible route to the target.

We are ML engineers — there must be a better way than trial and error! We can use an optimization algorithm to search systematically:

Define a loss function that takes as input the instance of interest, a candidate counterfactual, and the desired outcome.
Find the counterfactual that minimizes this loss using an optimization algorithm.

There are many methods for generating counterfactuals, and they differ primarily in how they define the loss function and which optimization strategy they use.

ℹ

Outcome loss plus distance

A common objective is:

$\arg\min_{x'}\lambda\left(\hat f(x')-y'\right)^2+d(x,x')$

The first term pushes the prediction toward target $y'$ . The distance term keeps counterfactual $x'$ close to original instance $x$ .

Diagram illustrating the optimization-based approach to finding counterfactual explanations — This is the sample of interest from the MNIST dataset and a generated counterfactual. We have also visualized the difference between them to show what pixels needed to change for this to be misclassified as a “6”.

Counterfactuals are easy to interpret and can work with black-box access. They also face the Rashomon effect: many different changes may reach the same target.

Good systems enforce immutability, feasibility, sparsity, and diversity. They should never recommend changing race, age, or a historical fact the person cannot alter.

Checkpoint

What makes a counterfactual actionable rather than merely close?

←PreviousPrototype-Based ExplanationsExample-Based Explanations Next→Influential InstancesExample-Based Explanations