What's a "Black Box"?
You may have heard machine learning or AI models, and specifically neural network architectures called "blackbox models. But what does that actually mean? And how does Explainable AI help?"
A black box has a human-understandable input and a human-understandable output, while the mapping between them is difficult for a person to follow. You understand a photo of a handwritten 2. You understand the output label 6. The layers of activations between the photo and the label do not tell a human-understandable story.
Walk Through a Black Box
Change an email, then move through the model one stage at a time. The input and final label are easy to read. The middle quickly turns into numbers that need interpretation.
Change a synthetic email and step from human-readable features through the model's hidden representation to its spam decision.
Adam Harley
Visualize a Neural Network
An interactive 3D visualization of a CNN processing handwritten digits in real time.
Watch activations flow through each layer of a CNN in real time.
The examples you observed above are with very simple neural networks containing only a few layers and a few dozen parameters. In many deployed neural networks (which you interact with daily -- from Netflix recommendations to directions in Google Maps to LLM chatbots), there are millions, billions, or even trillions of parameters!
Which situation best describes a black-box problem?