Faithfulness
An explanation is faithful when the factors it names are factors the model actually uses. This is a stronger demand than looking plausible to a person.
Faithfulness is tested through interventions: remove, insert, replace, or alter the alleged evidence and measure the model response.
Test the explanatory claim
If an explanation ranks features , deletion tests remove them in that order and track output degradation. Insertion tests add them back to a baseline. Strong explanations change the output quickly in the predicted direction.
Delete Features in the Claimed Order
Remove sentiment tokens from most to least important. A faithful ranking should damage the positive prediction quickly.
Original positive score
0.930
Current score
0.930
Cumulative deletion drop
0.000
Interventions can create unrealistic inputs, so use more than one test. For text, deletion may break grammar. For images, masking can introduce artifacts.
Generated rationales need the same skepticism. Alter the stated reason and see whether the answer changes.
Why can a deletion test falsely make an explanation look faithful?