AI Vocabulary · 01 The Model
What is Training?
The one-time, very expensive process that sets a model's parameters by showing it vast amounts of data.
Training is how a model is made. It is shown enormous amounts of text, predicts what comes next, and its parameters are adjusted a tiny bit after every guess. After trillions of repetitions, it predicts well. A later phase teaches it to follow instructions and behave helpfully.
Training happens once, before release, and is done by the model maker. It costs a lot of money, data and energy. Using the finished model is a different step: inference.
Whatever is in the training data shapes the model, including gaps and bias. And training stops at a date, which becomes the model's knowledge cutoff.
Why it matters for you
You are almost never training a model. When people say "we'll train the AI on our data", they usually mean giving it your documents as context, which is faster and cheaper. Real retraining is fine-tuning, and it is rarely the first step.
Own it
Ask vendors whether your inputs are used to train their models, and how to opt out. Your data should improve your results, not someone else's product, unless you choose that.
Where it comes from
Machines that learn from examples date back to the 1950s, but training multi-layer networks was popularised in 1986, when David Rumelhart, Geoffrey Hinton and Ronald Williams published Learning representations by back-propagating errors in Nature. Backpropagation, nudging every weight in the direction that reduces the error, still powers model training today. Hinton later shared the 2024 Nobel Prize in Physics for foundational work on neural networks. Backpropagation
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.