AI Vocabulary · 01 The Model
What is Reinforcement Learning?
A way of training in which a system learns by trial and error which actions earn the most reward over time.
In reinforcement learning, a system is not shown the right answers. It acts, receives a reward or a penalty, and gradually learns which behaviour pays off in the long run. It is the same principle as training with rewards, just running millions of times.
It became known through games and robotics. In language models it shows up in post-training: in RLHF, people compare answers and their preferences become the reward. Reasoning models are also trained this way. OpenAI wrote in 2024 that its o1 model gets better with more reinforcement learning.
Why it matters for you
A system trained with rewards optimises exactly what is rewarded, not what was meant. If people tend to reward pleasant answers, a model can learn to tell them what they want to hear, which is one reason for sycophancy.
Own it
The same applies when you set goals for an agent or an automation: decide precisely what success means before you let a system chase it, and check the result with evals.
Where it comes from
The field draws on decades of research in psychology, control theory and computer science. Its standard textbook is Reinforcement Learning: An Introduction by Richard Sutton and Andrew Barto, second edition 2018, free to read online. Learning from human preferences as a reward was introduced by Paul Christiano and colleagues in 2017.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.