AI Vocabulary · 08 Rules & Responsibility

What is Explainability?

Being able to understand and explain why an AI system produced a particular output or decision.

Intermediate

Explainability is about the "why" behind an AI result. For simple systems, such as a rule or a scoring model, you can often trace exactly which factors led to an outcome. For large language models that is much harder: their behaviour emerges from billions of parameters, and the reasons a model gives for its answer are themselves generated text, not a reliable look inside.

In practice, explainability for business use often means something more modest: showing the sources an answer was based on, the inputs and rules used, and enough information for a person to check the result and for an affected person to understand a decision about them.

Why it matters for you

When AI influences decisions about people, such as hiring, credit or prices, "the system said so" is not an acceptable explanation, legally or ethically. High-risk AI uses come with explicit requirements here.

Own it

Design for checkability: ask for sources, keep logs of inputs and outputs, prefer simple, traceable methods where decisions affect people, and keep a person who can explain and override the result through human oversight.

Related