AI Vocabulary · 07 When It Goes Wrong
What is Model Drift?
When results change over time without you changing anything, because the model, the data or the world moved.
Model drift means an AI setup that used to work gradually behaves differently. There are a few causes. The model provider updates or replaces the model behind the same name. The data your system works with changes: new products, new customer language, new regulations. Or the world moves on past the model's knowledge cutoff.
Because changes are often small and spread out, drift is easy to miss until a customer or colleague complains.
Why it matters for you
An AI workflow is not "done" when it ships. Without monitoring, quality can erode quietly, and you find out late and from the wrong people.
Own it
Pin model versions where your provider allows it, keep a set of evals and rerun them regularly and after every provider update, and log real failures as new test cases. Owning your test set means you notice drift first.
Where it comes from
In machine learning, "concept drift" describes the world changing so that a trained model slowly becomes less accurate; the idea goes back to research in the 1980s. With hosted AI, a second kind appeared: the model behind the same name changes. A 2023 Stanford and Berkeley study, How Is ChatGPT's Behavior Changing over Time?, measured clear differences between versions of the same model only months apart.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.