Introduction and Challenges
Large language models combine billions of parameters, high-dimensional entangled representations, open-ended outputs, and almost no ground truth for what a good explanation should look like.
The methods in this unit still apply, but scale and evaluation become harder.
Four places to explain
LLM explanations can target fine-tuned model behavior, prompt influence, retrieved knowledge, or agents. Each target exposes different evidence and failure modes.
Choose the System Boundary
Click the component you want to explain. The evidence you need changes depending on whether the question concerns the model, prompt, retrieval system, or tools.
What is inside this boundary?
Embedding, ranking, and passages
Useful evidence
Scores, citations, and entailment
Start by naming the system boundary. A chatbot answer may depend on the base model, fine-tuning data, system prompt, user prompt, retrieved documents, tools, and post-processing.
“Explain the LLM” is too broad. “Which retrieved passage supported this claim?” is answerable.