Explainability
Explainability is how clearly an AI product shows why it produced an answer: sources, reasoning steps, confidence, or feature influence.
Users do not need to read model internals, but they need enough transparency to trust, verify, and correct outputs in high-stakes work.
What it means
Explainability covers citations, chain-of-thought summaries, highlight-to-source links, confidence labels, and plain-language “why this recommendation” copy.
Why designers should care
When explainability is missing, users treat fluent text as truth; when it is cluttered, they ignore it. The job is calibrated disclosure for the task risk level.
Example
A loan review assistant shows three cited policy clauses, a one-line rationale, and a “See full reasoning” expander auditors can export, while shoppers see only a short summary.
Common mistakes
- • Fake explainability: generic “Based on your data” with no actual source links.
- • Dumping raw model reasoning on every user regardless of task sensitivity.
- • Explainability that disappears when the model is wrong, eroding trust further.