AI Vocabulary · 01 The Model
What is Transformer?
The neural network design behind today's language models, built to weigh how every part of a text relates to every other part.
The transformer is a neural network architecture introduced by Google researchers in 2017 in the paper "Attention Is All You Need". Its key mechanism, attention, lets the model weigh how each token in the input relates to all the others, so it can connect a pronoun to the right name or a function call to its definition.
Transformers can be trained efficiently on huge amounts of data in parallel, which is what made modern large language models possible. The "T" in GPT stands for transformer.
Why it matters for you
The design explains some practical limits. Because every token is compared with every other one, very long inputs get expensive, and quality can drop as the context window fills up. Shorter, focused input usually works better than dumping everything in.
Own it
You will rarely choose an architecture yourself. What you do control is what you feed the model: keep context relevant, and you get better results for less money.
Where it comes from
The Transformer was introduced in June 2017 by eight Google researchers in the paper Attention Is All You Need. Built for machine translation, it replaced older word-by-word designs with attention alone, which made training faster and far easier to run in parallel. Its influence is easy to spot: the "T" in GPT stands for Transformer.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.