AI Vocabulary · 07 When It Goes Wrong

What is Hallucination?

When AI states something false with full confidence, because it produces plausible text, not checked facts.

Basic

A hallucination is an answer that sounds right but is wrong: an invented statistic, a quote nobody said, a source that doesn't exist, a function in a software library that was never written. The tone gives no warning. A made-up answer arrives in the same confident style as a correct one.

It happens because a large language model doesn't look facts up. It generates the most plausible next piece of text based on patterns from its training. Where its training data is thin, outdated or past its knowledge cutoff, plausible and true drift apart, and the model has no built-in signal that tells it the difference.

Hallucinations also happen when the right information is there but gets lost: in a very long conversation or a crowded context, the model can drift away from the documents you gave it and fill in details from its own patterns instead.

Why it matters for you

Hallucinations are the main reason AI output can't be published, sent to a client or used for a decision without checking. The risk is highest exactly where errors are expensive: numbers, legal statements, names, citations and anything recent. Treat AI output as a strong first draft from a fast assistant who sometimes makes things up, not as a source.

Own it

You can't switch hallucinations off, but you can design around them. Give the model your own sources to work from, for example with RAG, and ask it to cite them. Keep a human in the loop for anything that leaves the building. And use evals to measure how often your setup gets things wrong, instead of trusting a feeling.

Related