LIME

A loan model gives one applicant a 42% approval probability. The compliance team does not need a tour of the entire model. They need to know what happened near this applicant.

Local Interpretable Model-Agnostic Explanations, or LIME, creates nearby synthetic cases, asks the black box to score them, and fits a simpler model to that neighborhood. The explanation comes from the simple model's coefficients.

ℹ

The LIME objective

LIME chooses an explanation $g$ that balances local fidelity and simplicity:

$\arg\min_g L(f,g,\pi_x)+\Omega(g)$

$f$ is the black-box model, $L$ measures how closely $g$ matches $f$ near instance $x$ , $\pi_x$ weights nearby samples, and $\Omega(g)$ penalizes complexity.

Fit a Local Surrogate

Change the neighborhood around one loan application. Nearby synthetic cases receive more weight, which changes the line used to explain the black-box score.

Neighborhood width0.28

Instance of interest

income = 0.62

Local slope

0.760

The line explains the sampled neighborhood, not the entire model. Kernel width decides what “local” means.

Text inputs work differently from tabular data. There is no continuous feature axis to perturb. Instead, LIME treats each word as a binary feature: either present in the perturbed input or replaced with nothing. A random mask drops some words, the masked text is scored by the model, and the process repeats. A linear model fitted to those (mask, score) pairs assigns each word an attribution.

Perturb Text to Build a Local Explanation

Choose a review and a word-keep rate. Each sample randomly removes words from the original. The word-weight panel estimates which words push the sentiment score up or down.

Word-keep probability60%

Original input

Theproductqualityisexcellentandthedeliverywasfast.

Original sentiment score

0.680

Perturbed samples — click to inspect

Estimated word weights

The

-0.080

product

-0.027

quality

+0.070

excellent

+0.071

and

-0.033

the

-0.074

delivery

-0.090

was

-0.027

fast

+0.070

Text LIME treats each word as a binary feature: present or absent. Weights come from a ridge regression fitted to the perturbed samples and their scores.

The neighborhood is the hard part. A narrow kernel can create a fragile explanation from very few useful samples. A wide kernel may smooth over the local behavior you wanted to inspect. In high-dimensional tabular data, synthetic cases can also combine feature values that rarely occur together.

Use LIME as a local approximation with declared settings. Run it more than once, vary the kernel width, and check whether the explanation is the same.

Image inputs require a different kind of neighborhood. LIME groups nearby pixels into superpixels — coherent regions that share color and texture — then perturbs the image by masking some regions to gray. Each synthetic image is a binary mask over those regions, not a list of individual pixel changes. This keeps the perturbation space manageable and the synthetic images visually plausible.

Superpixels as LIME's Input Units

LIME groups nearby pixels into superpixels, then creates perturbed images by masking regions out. Click superpixels to toggle them on or off — each combination becomes one synthetic sample in LIME's neighborhood.

Color by importance

Hovered region

hover a region to inspect

Perturbation mask

111111111111

12 of 12 regions present

Regions on

12 / 12

Original score

0.86

Perturbed score

0.86

Superpixels reduce the feature space from millions of pixels to a few dozen regions. The perturbation mask tells LIME which regions are present or absent in each synthetic sample.

⚠

Kernel width is an open problem

The current Python implementation of LIME uses an exponential smoothing kernel. A smoothing kernel is a function that takes two data instances and returns a proximity measure. Kernel width determines how large the neighborhood is: a small width means an instance must be very close to influence the local model, while a larger width lets instances farther away contribute.

There is not yet a robust way to find the best kernel width, and this problem gets much worse in higher-dimensional feature spaces.

Checkpoint

Two analysts explain the same applicant with different LIME kernel widths and get opposite coefficient signs. What should they conclude?

Next→AnchorsLocal Explanations