XAI in LLM Knowledge Augmentation (RAG)
Retrieval-Augmented Generation adds a visible knowledge path: query, embedding, similarity search, retrieved documents, prompt, and answer.
Source grounding improves transparency because a user can inspect the material supplied to the model.
RAG Pipeline, Step by Step
Step 1 of 6
Build the vector database
Unstructured data — product descriptions, documents, images — is passed through an embedding model ahead of time. Each item's embedding (a vector) is stored alongside the original data it represents.
Vector database
User query → embedding model
User
Embed
Similarity search
Compare the query vector against every vector in the database. Rank database items by closeness to the query.
Prompt assembly
query: "Do you have black leather boots?"
context: "SKU 4471 — Black leather boots, waterproof, sizes 6–12"
LLM generation → response
LLM
Embedding model
—
Top match
—
Response grounded in
—
Retrieval trace
A RAG explanation should expose the query transformation, retrieved items and scores, passages placed in context, citations attached to claims, and the answer generated from that context.
Citation is not proof of use. A model can cite a relevant document and generate a claim the document does not support. Retrieval can also fail before generation begins.
Evaluate retrieval recall, ranking, passage relevance, citation entailment, and answer faithfulness as separate steps.
What does a correct citation establish, and what remains unknown?