AI Vocabulary · 07 When It Goes Wrong
What is Alignment?
The effort to make AI do what people intend and value. What exactly that means is still debated.
Alignment means bringing an AI system's behaviour in line with human intentions and values. The word is used widely, but not in one sense. Three common uses:
- Following intent: in OpenAI's 2022 InstructGPT paper, an aligned model does what its users mean; unaligned outputs are untruthful, toxic or unhelpful.
- A set of values: in a 2021 Anthropic paper, an aligned assistant is "helpful, honest, and harmless".
- A future problem: Anthropic also uses the term for advanced systems that might pursue goals against human interests, and says nobody yet knows how to train very powerful AI to be robustly helpful, honest and harmless.
In practice, today's alignment work mostly happens in post-training: human feedback, rules and tests that shape how a model behaves.
All three uses come from companies that build and sell models. Their framing of the problem is also a framing of their own work.
Why it matters for you
When a provider says its model is "aligned", ask: aligned with what, and tested how? A model can be polite and safe in general and still fail your specific values, or tell you what you want to hear (sycophancy).
Own it
Write down what good behaviour means for your use: tone, limits, what must never happen. Turn it into instructions, guardrails and tests. That is alignment you control.
Where it comes from
The sources are Training language models to follow instructions with human feedback by Long Ouyang and colleagues (OpenAI, March 2022), A General Language Assistant as a Laboratory for Alignment by Amanda Askell and colleagues (Anthropic, December 2021) and Anthropic's Core Views on AI Safety (March 2023).
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.