AI Vocabulary · 07 When It Goes Wrong
What is Non-Determinism?
The same question can get a different answer each time.
Ask a model the same thing twice and you may get two different answers, sometimes a different word, sometimes a different conclusion. That is because a model picks each token from a range of likely options, usually with some deliberate randomness, so that the text doesn't become repetitive. Small differences early in an answer grow into bigger ones later.
This is not a bug that can simply be switched off. It is part of how LLMs generate text.
Why it matters for you
One good result in a demo proves little. One bad result proves little too. For anything you rely on, test several runs and judge the spread, not a single answer. Retrying is a legitimate strategy.
Own it
If consistency matters, as in classification, extraction or compliance checks, use fixed instructions, clear output formats and evals that run many examples, so you measure reliability instead of hoping for it.
Where it comes from
Non-determinism is a classic computer science term: the same input does not always give the same output. For LLMs it was long blamed simply on sampling and floating-point maths. In September 2025, Horace He at Thinking Machines Lab published Defeating Nondeterminism in LLM Inference, showing that a big hidden cause is how servers batch requests together, and how to make outputs reproducible.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.