AI Vocabulary · 02 AI Infrastructure

What is GPU?

A graphics processor: a chip that runs thousands of simple calculations at once, which made it the workhorse of AI.

Basic

A GPU, graphics processing unit, was built to draw images on screens. That job means doing the same simple calculation for millions of pixels at the same time. Training neural networks needs exactly that: huge numbers of multiplications that can run in parallel.

A CPU, the main processor in every computer, is a generalist that handles many different tasks well. A GPU is less flexible but can run far more calculations at once. According to the Georgetown research centre CSET, GPUs dominated AI training from around 2017.

GPUs are now so central that they serve as a unit of measurement. Epoch AI counts data centre capacity in "H100 equivalents", after one of NVIDIA's data centre GPUs.

Why it matters for you

When you hear about chip shortages, export controls or AI companies raising billions, it is mostly about GPUs and other AI chips. Their price and availability flow straight into what you pay for AI.

Own it

For running small or open-weight models yourself, a single good GPU, or a laptop with a strong graphics chip, can be enough. Check what the model needs before buying hardware or cloud capacity.

Related