Data

What it is about

A language model learns by predicting the next piece of text, trillions of times over. The examples it learns from are its training data: web pages, books, code, scientific papers and, more and more, text written by other models. Before training, all of this is cut into tokens, small chunks of text that the model handles as numbers.

Data shapes a model in three phases. In pre-training it absorbs language and knowledge from a filtered copy of the web. In post-training people and other models show it which answers are helpful. When you use it, your own documents can reach it through RAG while the model itself stays unchanged. Each phase raises its own questions about quality, cost, consent and law, and courts on several continents are answering them right now.

Tokens: how text becomes something a model can learn

A model never sees letters or words. It sees tokens: frequent chunks of text, each mapped to a number. The common method, byte-pair encoding, comes from data compression. It starts with single characters and keeps merging the pairs that occur most often until a fixed vocabulary of word pieces is reached. Sennrich, Haddow and Birch introduced these subword units for machine translation in 2015, so that rare and unknown words can be built from known pieces.

OpenAI's rule of thumb for English: one token is about four characters or three quarters of a word, so 100 tokens are about 75 words (vendor guidance). Training data is counted the same way. DeepMind's Chinchilla model of 2022 learned from 1.4 trillion tokens, Meta's Llama 3 from about 15 trillion.

Tokenisation is fixed once, before training, and it treats languages unequally. Petrov, La Malfa, Torr and Bibi showed at NeurIPS 2023 that the same text translated into different languages can need up to 15 times as many tokens. Providers charge per token, and a model's context window is measured in tokens. So the same task in many languages other than English costs more, takes longer and fits less into one request.

Try it yourself: paste the same paragraph in English and in your language into a provider's token counter. The difference is your language surcharge. OpenAI's help centre itself notes that the ratio depends on the language.

From web crawl to training set

A training set is the result of many deliberate deletions, and most crawled text is thrown away. The two best-documented pipelines come from Meta (Llama 3) and Hugging Face (FineWeb, about 15 trillion tokens from 96 snapshots of the open archive Common Crawl). OpenAI, Anthropic and Google publish their filters in far less detail.

Many filters are judgement calls. FineWeb deliberately skipped a common rule that drops lines without a full stop at the end, because that rule alone would have removed about 30% of all tokens. The quality judges are often earlier models: Llama 3's classifiers learned from Llama 2's ratings, so machine judgement already shapes which human text a new model sees.

1. 01Crawl. A web crawler downloads public pages. Common Crawl's October 2026 archive holds 2.16 billion pages and 371 TiB of uncompressed data, with 830 million URLs never visited before. Llama 3's data reaches to the end of 2023, its knowledge cutoff.
2. 02Extraction. Software pulls readable text out of the HTML and drops menus and boilerplate. FineWeb re-extracted text from the raw archive files because the ready-made text files kept too much menu text; the cleaner version trained a better model.
3. 03Language detection. A small classifier (fastText) labels each page. FineWeb keeps pages with an English score of at least 0.65; Llama 3 recognises 176 languages and filters each one separately.
4. 04Rule-based filters. Rules remove spam, error logs, placeholder text and pages full of repeated phrases. Three custom FineWeb rules removed about 22% of tokens and raised benchmark scores by about 1%.
5. 05Deduplication. Deduplication removes copies. Llama 3 works on three levels: URLs, whole documents (with MinHash, a fast way to spot near-duplicates) and single lines that repeat more than six times per 30 million documents. In C4, an older dataset, one 61-word sentence appeared more than 60,000 times.
6. 06Quality scoring. A classifier rates every page. For FineWeb-Edu, Llama 3 70B scored 460,000 pages on a 0 to 5 scale of educational value, and a small model trained on these ratings judged the rest. Keeping pages with a score of 3 or more shrank 15 trillion tokens to 1.3 trillion at a cost of 6,000 H100 GPU hours, and the smaller set did much better on knowledge tests.
7. 07Safety and personal data. Llama 3 removes domains with large amounts of personally identifiable information (PII) and adult content. FineWeb uses a blocklist of adult sites and replaced email and IP addresses in its public release.
8. 08Mixing. The final recipe decides what the model is good at. Llama 3 used about 50% general knowledge, 25% maths and reasoning, 17% code and 8% other languages, chosen with small test models and scaling laws. A last phase on small sets of high-quality code and maths raised an 8B model's score on a school maths test by 24%, with negligible effect on the 405B model.

A surprise from FineWeb: removing duplicates across all 96 snapshots at once left 4 trillion tokens and trained a clearly worse model than removing them within each snapshot, which left 20 trillion. Cleaning has an optimum too.

How much data it takes, and what quality adds

In 2022 DeepMind's Chinchilla paper made data a scaling axis. For a fixed compute budget, model size and the number of training tokens should grow together; its table implies about 20 tokens per parameter, for example 1.5 trillion tokens for a model with 67 billion parameters. The authors added that larger datasets only help when the data is of high quality.

Today's open models train far beyond that ratio, because a small model that has seen more data is cheaper to run. By our calculation, Llama 3's 8B model saw about 1,900 tokens per parameter. Repeating good data also works: Muennighoff and colleagues found that reusing data for up to four passes (epochs) is almost as good as fresh data. Beyond that, extra compute adds less and less, as scaling laws predict.

Selection often beats volume. Microsoft's phi-1, with 1.3 billion parameters, learned from only about 7 billion tokens of textbook-quality web text and synthetic textbooks and solved 50.6% of the HumanEval coding benchmark (vendor research). The DCLM project trained a 7B model on a filtered set of 2.6 trillion tokens; across 53 tasks it matched Llama 3 8B with 6.6 times less compute.

Will the data run out? Epoch AI estimated in 2024 that the stock of usable public human text is around 300 trillion tokens. Its paper gives a median of 320 trillion with a wide range of 65 to 1,700 trillion and elsewhere rounds to about 400 trillion, because the figures measure slightly different things. Datasets grow about 2.4 times a year, so they would reach the stock between 2026 and 2032 (forecast). The largest disclosed open dataset, Qwen3's 36 trillion tokens, is about 11% of the median. High-quality text is becoming the bottleneck.

• Llama 2 (Meta, 2023): 2 trillion tokens according to the Llama 2 paper; the Llama 3 paper gives 1.8 trillion.
• DeepSeek-V3 (2024): 14.8 trillion tokens.
• Llama 3 (Meta, 2024): about 15 trillion tokens; the paper also names 15.6 trillion for the 405B model and leaves the two figures unreconciled.
• Qwen3 (Alibaba, 2025): 36 trillion tokens in 119 languages and dialects.
• OpenAI, Anthropic, Google: undisclosed. Press figures for GPT, Claude or Gemini are unverified.

All dataset sizes are the developers' own figures from their technical reports.

Post-training: the data that shapes behaviour

Pre-training produces a model that can continue any text. It has no sense yet of what a helpful answer looks like. Post-training adds that sense with far smaller, carefully made datasets. OpenAI's InstructGPT paper of 2022 showed the effect: people preferred the answers of a 1.3-billion-parameter model trained this way to those of the 175-billion-parameter GPT-3 (vendor research).

Since 2022 the training signal has moved from human labels towards signals generated by AI or by rules. Human work shifts to writing principles, designing checkable tasks and rating expert answers. Frontier labs keep the size of their preference datasets to themselves.

1. 01Supervised fine-tuning (SFT). People write example answers to prompts, and the model is fine-tuned on them. This teaches format and tone.
2. 02Preference data and RLHF. People compare two or more answers and pick the better one. A reward model learns these preferences, and the main model is trained with reinforcement learning to score well on it. This is RLHF, reinforcement learning from human feedback.
3. 03DPO. Direct preference optimisation (Stanford, 2023) learns from the same answer pairs with a simple classification step and skips the separate reward model and the reinforcement loop.
4. 04AI feedback (RLAIF). In Anthropic's Constitutional AI (2022), humans supply only a list of principles. The model critiques and revises its own answers, and an AI judges pairs of answers to train the reward model.
5. 05Verifiable rewards (RLVR). The model is rewarded only when its answer can be checked automatically, for example a maths result that matches or code that passes its tests. The Allen Institute for AI named the method in its open Tulu 3 recipe (2024). DeepSeek-R1 (Nature, 2025) reports that reasoning can be learned through reinforcement learning alone, without human-written reasoning examples, a basis of today's reasoning models.

Aulory reading of the sources: your own data is worth most here. A few thousand clean, domain-specific examples or test cases do far more for fine-tuning and evaluation than they could ever do in pre-training.

Synthetic data, model collapse and contaminated tests

Synthetic data is text that models generate for training. It is now standard: Qwen3 was pre-trained partly on textbooks, questions and answers, instructions and code written by earlier Qwen models, and phi-1 used a billion tokens of synthetic textbooks. Ordinary web data is already mixed: researchers found machine-generated and machine-translated text in the C4 dataset as early as 2021.

The risk has a name: model collapse. Ilia Shumailov and colleagues showed in Nature in 2024 that models trained over generations on their own output lose the rare cases of the original data first, and the damage stays. A follow-up by Gerstgrasser, Schaeffer, Koyejo and others traced collapse to replacing real data with synthetic data; when synthetic data was added on top of the real data, the error stayed bounded.

How much synthetic text is in today's frontier models? No credible source states a share, and we publish none. A controlled study by Meta-affiliated researchers (2025) found that mixing about one third rephrased synthetic text with two thirds web text reached the same quality 5 to 10 times faster, tested on models of up to 3 billion parameters.

Data also decides whether test results mean anything. A benchmark measures ability only if its questions were absent from training, and contamination is documented. Over 4% of the validation sets of standard datasets overlap with training data (Lee and colleagues). On GSM1k, a fresh set of school maths problems matched to the popular GSM8k, leading models lost up to 8% accuracy (NeurIPS 2024); many frontier models showed little overfitting.

Practical consequence: build your own held-out test set from real cases, often called a golden set, and keep it out of any data you send for training. Use it for evals before you choose a model.

Whose language, whose world: bias and coverage

Bias enters at three points: what is crawled, what filters delete and how text is cut into tokens. The raw web is already skewed. In Common Crawl's October 2026 archive, 42.6% of pages are primarily in English, followed by Russian (6.3%), German (5.8%), Japanese (4.9%), French (4.8%), Spanish (4.7%) and Chinese (4.4%). Hindi accounts for 0.21%, Swahili for 0.01%.

Training sets can be even more skewed. Meta reported that Llama 2's pre-training data was 89.7% English and warned that the model would work best in English. Llama 3 used 8% multilingual tokens; Qwen3 covers 119 languages and dialects. An audit of nearly 4,000 public datasets by the Data Provenance Initiative found that their geographic and language representation has barely improved since 2013.

Filters add their own bias. Blocklists of offensive words in C4 removed text about minority individuals at higher rates (Dodge and colleagues, 2021). Abeba Birhane's team found that hateful content rose by nearly 12% when an image-text dataset grew from 400 million to 2 billion pairs. Tokenisation then adds a surcharge, with up to 15 times as many tokens for the same text in some languages.

Others are closing gaps. Cohere's Aya project had fluent speakers from 119 countries write instruction data in 65 languages. Hugging Face's FineWeb2 applies one adaptable cleaning pipeline to more than 1,000 languages. For high-risk AI, the EU AI Act requires training, validation and test data to be relevant and sufficiently representative, including for the groups affected (Article 10, recently amended; check the current text).

What models remember, and how data can be poisoned

Models store patterns, and sometimes whole passages. This is called memorisation. Carlini and colleagues extracted hundreds of verbatim sequences from GPT-2 in 2020, including names, phone numbers, email addresses and code, some of which appeared in only one training document. In 2023 a team including Google DeepMind researchers extracted gigabytes of training data from open and closed models; one attack made ChatGPT emit training data about 150 times more often. Alignment training did not remove memorisation.

Duplicates raise the risk: deduplication cut verbatim memorised output about tenfold in one study. For privacy law this matters directly. The European Data Protection Board (Opinion 28/2024) says whether a model counts as anonymous must be judged case by case, and that unlawfully processed training data can taint later use under the GDPR. France's regulator CNIL (February 2025) adds that people must be informed when their data may be memorised and asks developers to build privacy in from the start, because rights such as erasure are hard to exercise on a trained model.

Data poisoning is the deliberate planting of manipulated data. In a 2025 study by Anthropic, the UK AI Security Institute and the Alan Turing Institute, 250 poisoned documents, about 0.00016% of the training tokens, were enough to plant a backdoor in models from 600 million to 13 billion parameters. The backdoor only made the model output gibberish after a trigger word, and whether the result holds for larger models or more harmful behaviour is open. OWASP lists poisoning across pre-training, fine-tuning and embeddings and recommends tracking data provenance, vetting data vendors and versioning datasets.

Keep secrets, passwords and unnecessary personal data out of anything that might be used for training. Once trained in, they are practically impossible to remove, which makes this a form of data leakage with no undo. Anthropic's own terms state that data already used for training stays in those models.

Rights, consent and documentation

Who may train on what is decided region by region (see the table below). In the US, courts weigh fair use, a flexible exception in copyright law, case by case. In June 2025 a judge held in Bartz v Anthropic that training on lawfully bought books was fair use, while keeping a permanent library of pirated copies was not; the case settled for $1.5 billion, about $3,000 per book. On 29 September 2026 the Third Circuit became the first appeals court to rule: in Thomson Reuters v Ross it rejected fair use for a non-generative legal research tool built on 2,243 headnotes and saw harm to an emerging market for licensing headnotes as AI training data. It expressly left generative AI open. More in copyright and AI.

The EU allows text and data mining unless rights holders reserve their rights in machine-readable form. Since August 2025 it also requires providers of general-purpose AI models to publish a summary of their training content on a Commission template. In November 2025 a Munich court ruled that song lyrics memorised in a model are a reproduction (GEMA v OpenAI, not final). Prices are forming in parallel: Reddit's deals with Google and OpenAI are reported at about $60 million and $70 million a year (press figures, unconfirmed by the companies).

Consent on the web runs mainly through robots.txt, a voluntary file that tells a crawler what to skip. How many sites block AI crawlers depends on the sample: 5.7 to 5.9% of some 140 million websites (Ahrefs, an SEO vendor), 12.7% of 11,100 top landing pages by name and 39.9% when blanket blocks of all bots count (Zyte, a scraping company), and over 28% of the most actively maintained sources behind the C4 dataset (Longpre and colleagues, 2024). Since July 2025 Cloudflare blocks AI crawlers by default for new customers. A finer vocabulary is in the works at the IETF: draft 08 of September 2026 separates AI training, use as input to a model, and search. It is still a draft and states that it does not reflect working-group consensus.

Documentation makes data checkable. Timnit Gebru and co-authors proposed in 2018 that every dataset come with a datasheet; Google's Data Cards (2022) are a structured version in practical use, and models get model cards. The EU training-content summary, with its template of 24 July 2025, is the first form required by law.

This overview is general information. Laws and cases move fast, so check the current status with a lawyer before you rely on it.

The people behind the data

Behind every assistant sits human work: people write example answers, rate outputs, label images and moderate disturbing content. This is data labelling, and it feeds RLHF and every other post-training method. Market research firm Grand View Research put the market at $3.8 billion in 2024 and expects more than $17 billion within five years (estimate, reported by AFP).

Nobody counts these workers precisely. The World Bank estimated 154 to 435 million online gig workers worldwide in 2023, across all kinds of tasks; the high figure is contested. Informal government estimates count more than 2 million Filipinos in online crowdwork, and India's IT industry body NASSCOM sees a potential annotation workforce of up to 1 million. In Kenya, the Data Labelers Association has about 800 members.

Workers describe insecurity. In March 2024 Remotasks, a Scale AI subsidiary, closed Kenyan workers' accounts after their withdrawals had been frozen since December, reports organiser Mophat Okinyi. The Washington Post interviewed 36 workers in the Philippines in 2023; all but two reported delayed, reduced or cancelled pay. Scale says its rates meet or exceed local minimum wages. In July 2026 Kenya's ICT ministry was drafting rules on minimum pay and mental health care for annotators and moderators.

• Kenya (2023): $1.32 to $2 an hour take-home for workers labelling for OpenAI via Sama; Sama stated $1.46 to $3.74 after tax, while OpenAI paid Sama $12.50 an hour (TIME).
• Philippines (2023): minimum wage of $6 to $10 a day; on some Remotasks projects the pay per task fell from $10 to under one cent (Washington Post).
• Venezuela and Colombia: a little over 90 cents an hour on average on Remotasks, by Scale's own account (2022); 5 to 25 US cents per task for one Venezuelan worker in Colombia (AFP, 2025).
• India (2026): Rs 10,000 to 15,000 a month at annotation firms in Noida, Rs 17,000 after four years (ThePrint).
• Expert raters: $30 to $50 an hour for specialists on Outlier, a Scale platform; Anthropic says its subcontractors must pay at least $16 an hour (company statement, via AFP).
• A counter-model: Karya in India guarantees at least $5 an hour and passes all resale revenue from the data back to the workers (company figures reported by TIME).

All pay figures come from workers, companies or reporters and are unaudited. They span roughly a factor of 100, from cents per task to $50 an hour.

Your data: RAG, fine-tuning and what providers keep

For company knowledge, the usual route is RAG, retrieval-augmented generation. The system first searches your documents and hands the relevant passages to the model together with the question. The search runs on embeddings, lists of numbers that capture meaning, stored in a vector database so that passages close in meaning to the question can be found. The idea goes back to a 2020 paper from Facebook AI Research, UCL and NYU.

Fine-tuning changes the model itself. In a 2023 comparison, retrieval beat fine-tuning at adding knowledge, both for facts the model had seen before and for new ones; models struggled to learn new facts through fine-tuning. With RAG your documents still travel to the provider with every question, so the provider's data terms apply.

Those terms differ by tier. The following are vendor self-reports and can change. OpenAI says it does not use business and API data for training by default, keeps API data for up to 30 days for abuse monitoring and offers zero retention on request for eligible services. Anthropic lets Free, Pro and Max users choose whether their chats train models, with five years of retention if they agree and 30 days if they decline; business tiers and the API are excluded. Google uses content from the free Gemini API tier to improve its products, with human reviewers, and says it does not use prompts from paid services for that. Courts can override deletion: in NYT v OpenAI a judge ordered 20 million de-identified ChatGPT logs to be produced.

• RAG: facts that change, answers with sources, access rights per user. The data stays outside the model and can be updated or deleted.
• Fine-tuning: consistent tone or format, classification, a narrow task repeated many times, often on a small language model.
• Embeddings alone: semantic search, grouping or deduplicating your own documents, with no text-generating model involved.
• Both together: a fine-tuned model that answers from retrieved documents.

1. 01Choose the tier. Use business or API tiers for company data and keep it out of free consumer accounts.
2. 02Check the settings. Look up the training setting and the retention period of every tool, and write them down.
3. 03Put it in the contract. Ask for zero retention or a fixed data region where available and fix it in a data processing agreement. Where data is stored and which law applies is the core of data sovereignty.
4. 04Send the minimum. Remove personal data and secrets before they go into a prompt.
5. 05Treat your data as an asset. Keep your best data as RAG sources, fine-tuning examples and a private test set that you control.
6. 06Document provenance. Record source and rights for any data you train on, in a short datasheet. This prepares you for EU AI Act duties if you use AI in high-risk areas or become a model provider.

The decision rule and the six steps are Aulory's synthesis of the sources on this page.

What the layer is made of

Key figures, with source and date

• 36 tn tokensused to pre-train Qwen3 in 119 languages, the largest disclosed open dataset. OpenAI, Anthropic and Google publish no figures.Alibaba, Qwen3 Technical Report · 14 May 2025 · vendor
• 2.16 bn pagesin Common Crawl's October 2026 archive, 371 TiB uncompressed. Most training sets build on this open web archive.Common Crawl · 21 Oct 2026 · neutral
• 42.6%of pages in the latest Common Crawl are primarily English. Hindi has 0.21%, Swahili 0.01%.Common Crawl statistics · Oct 2026 · neutral
• 6.6xless compute: a 7B model trained on the filtered DCLM set of 2.6 tn tokens matched Llama 3 8B across 53 tasks.DataComp-LM (Li et al.) · 17 Jun 2024 · research, preprint
• 2026 to 2032is when training sets are projected to reach the stock of public human text, a median of about 320 tn tokens.Epoch AI · 4 Jun 2024 · neutral, forecast
• over 80%of the source material in widely used text, speech and video datasets carries non-commercial restrictions. Fewer than a third of the datasets themselves have restrictive licences.Longpre et al., Data Provenance Initiative · 19 Dec 2024 · research, preprint
• 250poisoned documents, about 0.00016% of training tokens, planted a backdoor in models of up to 13 bn parameters.Anthropic, UK AISI, Alan Turing Institute · 9 Oct 2025 · vendor with public institutes
• $3.8 bndata labelling market in 2024, expected to pass $17 bn within five years. Pay ranges from cents per task to $50 an hour.Grand View Research via AFP · 16 Oct 2025 · market research, secondary source
• $1.5 bnsettlement between Anthropic and book authors, about $3,000 per book, the largest price tag on training data so far.Jurist · Jul 2026 · secondary source

Regions compared

Rules on training data follow different logics. The US leaves it to courts case by case. The EU, Japan and Singapore have legal exceptions with conditions or opt-outs. India and Brazil discuss licensing and payment models, and the UK has left the question open. China supervises training data through state rules. Many of these rules are drafts or under appeal, and comparable figures on data work and languages exist only for some regions.

Sources: LawNext (12 Oct 2026), Jurist (Jul 2026), AI Lawsuit Tracker, European Commission (Apr 2026), Bird & Bird, CMS, Cross-Border Data Forum (19 Mar 2026), Irish DPC (21 May 2025), DLA Piper (19 Mar 2026), Burges Salmon (19 Dec 2025), China Law Translate, Digital Policy Alert, APAA (Oct 2024), IPOS (2026), Takshashila (Dec 2025), ThePrint (1 Sep 2026), GoLegal (21 Jul 2026), Business Daily Africa (24 Jul 2026), TIME (18 Jan 2023), casrai.org and CISAC on Brazil, MIT Technology Review (20 Apr 2022), AFP via Casilli (16 Oct 2025), Common Crawl statistics. Many entries rely on law-firm or press summaries of the original. Not legal advice.

Examples

• Common Crawl is the open web archive that most training datasets build on.
• FineWeb from Hugging Face filters about 15 trillion tokens of high-quality text out of it.
• Alibaba trains Qwen on web text, text extracted from PDFs and synthetic text from earlier models.
• Scale AI and Surge AI connect companies with people who label data and rate answers.

Why it matters

Data determines what a model knows, which languages it serves well and where it fails. Filtered data beats larger raw data, and the web's mix of languages and voices shows up in every answer. As public human text runs short and fills with machine-written text, original human data gains value, including your customer conversations and your experts' decisions.

Data also determines what is legal and what it costs. Courts on several continents are putting prices on training material, regulators ask how personal data got into models, and the people who label data are organising. For you, two questions follow: what happens to the data you give a model, and which of your own data is worth protecting and using.

Common misconceptions

• Every prompt I type trains the model.

  By default, OpenAI and Anthropic exclude business tiers and APIs from training, and Google says the same for paid Gemini services; some free and consumer tiers train on chats or let you choose (vendor statements).

• Fine-tuning is how you teach a model your company knowledge.

  In a direct comparison, retrieval (RAG) added knowledge better than fine-tuning (Ovadia and colleagues, 2023). Fine-tuning shapes behaviour, tone and format.

• More data always makes a better model.

  The Chinchilla authors themselves say larger datasets help only with high-quality data; filtered sets such as FineWeb-Edu and DCLM beat larger raw sets.

• Models keep only abstract knowledge.

  They can reproduce training text word for word, including personal data (Carlini and colleagues, 2021; Nasr and colleagues, 2023).

• Synthetic data ruins models.

  Collapse appeared when synthetic data replaced real data; added on top of real data, the error stayed bounded, and curated synthetic textbooks helped Microsoft's phi-1 (Gerstgrasser and colleagues, 2024).

• Benchmark scores are objective.

  Test questions leak into training data. On a fresh maths set, leading models lost up to 8% accuracy (GSM1k, NeurIPS 2024).

• Opting out later removes my data from the model.

  Anthropic states that data already used for training stays in those models, and France's CNIL notes that erasure is hard to exercise on a trained model.

Perspectives from leading experts

Ten perspectives from research, industry, data workers and an Indigenous community organisation. Each summary paraphrases a published work, linked below it. Vendors are marked as such.

• Shayne Longpre and co-authors

  Team lead of the Data Provenance Initiative, a volunteer collective of independent and academic researchers, North America · academic

  An audit of nearly 4,000 public datasets from 1990 to 2024 found that fewer than a third carry restrictive licences, while over 80% of the underlying source material has non-commercial restrictions. A dataset's licence label can therefore understate the real limits. Despite more languages and countries, geographic and language representation has barely improved since 2013, a skew the authors call Western-centric.

  Bridging the Data Provenance Gap Across Text, Speech and Video · Dec 2024

• Timnit Gebru and co-authors

  Founder and executive director of the DAIR Institute, formerly co-lead of Google's Ethical AI team, USA · academic

  Machine-learning datasets lack a standard way of documentation, and a mismatch between a dataset and its use can cause harm. Borrowing from electronics, the authors propose that every dataset ship with a datasheet covering motivation, composition, collection, preprocessing and labelling, uses, distribution and maintenance. Writing it by hand makes creators think deliberately and gives users what they need to choose the right data.

  Datasheets for Datasets (Communications of the ACM) · Dec 2021

• Abeba Birhane and co-authors

  Leads the AI Accountability Lab at Trinity College Dublin, Ireland · critical research

  Comparing the image-text datasets LAION-400M and LAION-2B, the team found that hateful content rose by nearly 12% as the dataset grew. Filtering on image-based NSFW scores alone left hateful, targeted and aggressive alt-text in place. Scaling up scraped data scales its problems too.

  Into the LAION's Den: Investigating Hate in Multimodal Datasets · Nov 2023

• Andrew Ng

  AI researcher and entrepreneur, leading advocate of data-centric AI, USA · vendor

  He defines data-centric AI as the discipline of systematically engineering the data an AI system needs. In industries without giant datasets, clean and well-chosen data matters most, and around 50 carefully engineered examples can be enough. Teams should find and fix inconsistent labels and collect more data only where the model is weak.

  Andrew Ng: Unbiggen AI (IEEE Spectrum) · 9 Feb 2022

• Sara Hooker and co-authors

  Head of Cohere Labs until September 2025, now co-founder of the start-up Adaption Labs, Canada and USA · vendor and academic

  Instruction-tuning data was almost entirely English. The Aya project had fluent speakers write natural prompts and answers, producing a human-curated dataset in 65 languages and a larger collection of 513 million examples in 114 languages, with contributors from 119 countries. The authors present it as a case study in participatory research.

  Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning · Feb 2024

• Ilia Shumailov and co-authors

  Lead author of the research on model collapse, UK · academic

  When models learn from web content increasingly written by other models, the rare, low-probability parts of the original data vanish first, and the damage cannot be undone. The effect appears in simple models and in language models and applies to generative models in general. As synthetic content spreads, genuine human data becomes more valuable.

  The Curse of Recursion: Training on Generated Data Makes Models Forget · May 2023

• Adrienne Williams, Milagros Miceli and Timnit Gebru

  Authors writing with the DAIR Institute on data work · critical research

  Talk of sentient AI distracts from the millions of underpaid, heavily surveilled workers who make today's systems run. The authors cite data workers earning as little as $1.46 an hour after tax and moderators in Kenya reportedly given 50 seconds per decision on violent videos. They want support for transnational worker organising at the centre of ethical AI.

  The Exploited Labor Behind Artificial Intelligence (Noema) · 13 Oct 2022

• Mophat Okinyi

  Former content moderator and data labeller, founder and CEO of Techworker Community Africa, Kenya · advocacy and communities

  In December 2023 the payment platform AirTM froze Remotasks workers' earnings, and on 9 March 2024 Remotasks shut their accounts for good, affecting thousands of workers in African countries. Workers demand refunds of the frozen money, rules for gig work, mental health support for those reviewing graphic content, an end to pay differences by geography and fairer automated quality scores.

  Impact of Remotasks' Closure on Kenyan Workers (Data Workers' Inquiry) · 2024

• Keoni Mahelona and co-authors, Te Hiku Media

  Māori non-profit media organisation that also sells te reo Māori language services, Aotearoa New Zealand · advocacy and communities

  They argue that OpenAI's Whisper, trained on 680,000 hours of web audio, likely includes te reo Māori and ʻōlelo Hawaiʻi scraped without consent, repeating colonial extraction. They see data as a taonga to be looked after, and say the communities it came from should decide how it is used. Fine-tuned on about 125 hours of their own data, Whisper reached a 38% word error rate, against 53% for their earlier system.

  OpenAI's Whisper is another case study in Colonisation (Papa Reo) · 24 Jan 2023

• Manu Chopra

  CEO of Karya, a data and evaluation company in Bengaluru, India · vendor

  According to TIME's report, Karya guarantees workers at least $5 an hour, nearly 20 times India's minimum wage, and returns all resale revenue from their data to them. About 30,000 rural workers had been paid around $800,000 at the time, including royalties. Chopra calls today's wages in data work a failure of the market.

  TIME feature on Karya · 2023

Terms for this layer

• Training DataThe text, images, code and other examples a model learns from. What goes in shapes what comes out.
• TokenThe small chunk of text a model reads and writes. Price, speed and memory limits are all counted in tokens.
• Pre-TrainingThe first and most expensive phase of training, in which a model learns general patterns from huge, broad data.
• Post-TrainingEverything that happens after pre-training to turn a raw base model into a helpful, safer assistant.
• ParametersThe billions of numbers inside a model, set during training. Everything the model knows is stored in them.
• Context WindowThe maximum amount of text a model can take in at once. When it is full, something has to go.
• Web CrawlAutomatically downloading huge numbers of public web pages. The raw material of almost every training dataset.
• Knowledge CutoffThe date after which a model learned nothing new. Anything later must be given to it.
• DeduplicationRemoving copies and near-copies from training data, so a model learns more and memorises less.
• Scaling LawsThe observation that a model's error falls predictably as model size, data and computing power grow.
• BenchmarkA standard test used to compare models. Useful for orientation, but no substitute for testing on your own tasks.
• Synthetic DataTraining examples generated by a model or a program instead of written or recorded by people.
• Fine-TuningTraining an existing model further on your own examples to change its behaviour. Rarely the first step.
• Reinforcement LearningA way of training in which a system learns by trial and error which actions earn the most reward over time.
• RLHFReinforcement learning from human feedback: people rank a model's answers, and the model is trained to give the kind of answers people prefer.
• Reasoning ModelA model that generates intermediate working steps before its final answer, trading time and tokens for better results on hard problems.
• Model CollapseWhat happens when models are trained again and again on AI-generated data: the rare cases disappear, and the damage stays.
• Benchmark ContaminationWhen test questions end up in a model's training data, so its benchmark score overstates its ability.
• EvalsRepeatable tests that show whether your AI setup actually works, and whether a change made it better or worse.
• BiasSystematic unfairness in AI output, inherited from data or design choices.
• High-Risk AI SystemAI used in sensitive areas like hiring, credit or education, which faces the strictest EU rules.
• EU AI ActThe EU's law for AI, sorting uses by risk and setting duties for those who build and deploy AI.
• MemorisationWhen a model stores pieces of its training data and can repeat them word for word, including personal data.
• GDPRThe EU's data protection law. It applies whenever AI tools process personal data about people in the EU.
• Data PoisoningPlanting manipulated data in a model's training or search sources to change how it behaves.
• Data LeakageConfidential information ending up where it shouldn't: in a prompt, a log, or another user's answer.
• Copyright & AIThe open legal questions around training AI on protected works and who, if anyone, owns what AI generates.
• Text and Data MiningAutomatically analysing large amounts of text and data. In many countries a legal exception that covers AI training, with conditions.
• General-Purpose AI ModelThe EU AI Act's term for broadly capable models, like large language models, that can be used for many different tasks.
• robots.txtA small text file on a website that tells crawlers, including AI crawlers, which pages they should skip.
• Data CardA short, structured document that says where a dataset comes from, what is in it and what it should be used for.
• Model CardA short document that comes with an AI model and states what it is for, how it was tested and where it falls short.
• Data LabellingPeople tagging, writing and rating data so models can learn from it. The hidden human work behind AI.
• Retrieval-Augmented GenerationLooking up relevant documents first and handing them to the model, so it answers from your sources.
• EmbeddingsText turned into lists of numbers that capture meaning. The basis of semantic search and RAG.
• Vector DatabaseA database that stores embeddings, so you can search your documents by meaning instead of exact keywords.
• Training on Your DataWhen a provider uses what you type or upload to improve its future models. Often a setting, sometimes a contract term.
• Small Language ModelA compact language model that runs fast and cheaply, often on a laptop or phone, at the cost of some capability.
• Data Processing AgreementThe contract that sets out how a provider may handle personal data on your behalf. Required under the GDPR.
• Data SovereigntyKeeping control over where your data lives, who can access it and under which laws.
• AlignmentThe effort to make AI do what people intend and value. What exactly that means is still debated.
• ComputeThe computing power, and the chips and software behind it, that AI needs to be trained and to run.
• APIThe programmable connection to a model provider. You pay per token and keep more control than in a chat app.
• PromptEverything you send to a model to get a result: your question, instructions, examples and material.
• Data ResidencyWhere your data is physically stored and processed. It is one part of data sovereignty.

Sources

1. Language Model Tokenizers Introduce Unfairness Between Languages, Petrov, La Malfa, Torr, Bibi (NeurIPS 2023), 17 May 2023
2. The Llama 3 Herd of Models, Meta (vendor), 31 Jul 2024
3. The FineWeb Datasets, Penedo et al., Hugging Face, 25 Jun 2024
4. DataComp-LM: In search of the next generation of training sets, Li et al., 17 Jun 2024
5. Deduplicating Training Data Makes Language Models Better, Lee et al., 14 Jul 2021
6. Training Compute-Optimal Large Language Models (Chinchilla), Hoffmann et al., DeepMind (NeurIPS 2022), 29 Mar 2022
7. Textbooks Are All You Need (phi-1), Gunasekar et al., Microsoft (vendor), 20 Jun 2023
8. Qwen3 Technical Report, Alibaba (vendor), 14 May 2025
9. DeepSeek-V3 Technical Report, DeepSeek (vendor), 27 Dec 2024
10. Will we run out of data? Limits of LLM scaling based on human-generated data, Villalobos et al., Epoch AI, 4 Jun 2024
11. October 2026 crawl archive now available, Common Crawl, 21 Oct 2026
12. Language statistics of the crawls, Common Crawl, Oct 2026
13. Training language models to follow instructions with human feedback (InstructGPT), Ouyang et al., OpenAI (vendor), 4 Mar 2022
14. Direct Preference Optimization, Rafailov et al., Stanford, 29 May 2023
15. Constitutional AI: Harmlessness from AI Feedback, Bai et al., Anthropic (vendor), 15 Dec 2022
16. Tulu 3: Pushing Frontiers in Open Language Model Post-Training, Lambert et al., Allen Institute for AI, 22 Nov 2024
17. DeepSeek-R1 (Nature 645, 2025), DeepSeek (vendor), 22 Jan 2025
18. AI models collapse when trained on recursively generated data, Shumailov et al., Nature, via Oxford OATML, Jul 2024
19. Is Model Collapse Inevitable?, Gerstgrasser, Schaeffer, Koyejo et al., 1 Apr 2024
20. Demystifying Synthetic Data in LLM Pre-training, Kang et al., 3 Oct 2025
21. Documenting Large Webtext Corpora: A Case Study on C4, Dodge et al., 18 Apr 2021
22. A Careful Examination of LLM Performance on Grade School Arithmetic (GSM1k), Zhang et al. (NeurIPS 2024), May 2024
23. Bridging the Data Provenance Gap Across Text, Speech and Video, Longpre et al., 19 Dec 2024
24. Consent in Crisis: The Rapid Decline of the AI Data Commons, Longpre et al., 20 Jul 2024
25. Extracting Training Data from Large Language Models, Carlini et al., 14 Dec 2020
26. Scalable Extraction of Training Data from (Production) Language Models, Nasr, Carlini et al., 28 Nov 2023
27. AI and GDPR: new recommendations, CNIL (regulator), 7 Feb 2025
28. A small number of samples can poison LLMs of any size, Anthropic, UK AISI, Alan Turing Institute, 9 Oct 2025
29. LLM04:2025 Data and Model Poisoning, OWASP, 5 May 2025
30. Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, Ovadia et al., 10 Dec 2023
31. No more than an ordinary copyright case: Third Circuit unseals Thomson Reuters v Ross opinion, LawNext, 12 Oct 2026
32. Template for the public summary of training content, European Commission, 24 Jul 2025
33. AI Preferences vocabulary, draft 08, IETF, 14 Sep 2026
34. AI bot block rates, Ahrefs (vendor), 21 May 2025
35. State of Web Access 2026: robots.txt and AI agents, Zyte (vendor), 2026
36. New report by AFP on data workers, Antonio Casilli, reproducing AFP, 16 Oct 2025
37. OpenAI used Kenyan workers on less than $2 per hour, TIME, 18 Jan 2023
38. Enterprise privacy, OpenAI (vendor), 8 Jan 2026
39. Updates to our consumer terms, Anthropic (vendor), 28 Aug 2025
40. Gemini API additional terms of service, Google (vendor), 28 Apr 2026
41. Neural Machine Translation of Rare Words with Subword Units, Sennrich, Haddow, Birch (ACL 2016), 31 Aug 2015
42. What are tokens and how to count them, OpenAI Help Center (vendor, undated page, accessed October 2026), Oct 2026
43. Scaling Data-Constrained Language Models, Muennighoff et al., 25 May 2023
44. Llama 2: Open Foundation and Fine-Tuned Chat Models, Meta (vendor), Jul 2023
45. Will we run out of data? (blog summary), Epoch AI, 6 Jun 2024
46. FineWeb2: One Pipeline to Scale Them All, Penedo et al., Hugging Face, 26 Jun 2025
47. Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning, Singh et al., Cohere For AI, 9 Feb 2024
48. Into the LAION's Den: Investigating Hate in Multimodal Datasets, Birhane et al., 6 Nov 2023
49. Article 10: Data and data governance (amended text, secondary reproduction), artificialintelligenceact.eu, 2026
50. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Lewis et al., 22 May 2020
51. Data Cards: Purposeful and Transparent Dataset Documentation, Pushkarna, Zaldivar, Kjartansson, Google (vendor), 3 Apr 2022
52. Datasheets for Datasets, Gebru et al., Communications of the ACM, Dec 2021
53. Content Independence Day: no AI crawl without compensation, Cloudflare (vendor), 1 Jul 2025
54. Reddit weighs cutting Google AI access as $60m deal expires, AI Weekly (secondary), 22 Jul 2026
55. AI boom overseas workers (Washington Post), Stars and Stripes, 30 Aug 2023
56. How the AI industry profits from catastrophe, MIT Technology Review, 20 Apr 2022
57. AI is creating lakhs of low-paying jobs in Gurugram, Noida, ThePrint, 1 Sep 2026
58. Working Without Borders: The Promise and Peril of Online Gig Work, World Bank, 2 Oct 2023
59. AI giants face new minimum pay and mental healthcare rule in Kenya, Business Daily Africa, 24 Jul 2026
60. Impact of Remotasks' Closure on Kenyan Workers, Mophat Okinyi, Data Workers' Inquiry, 2024
61. Data Workers' Inquiry, DAIR, Weizenbaum Institute, TU Berlin (undated page, accessed October 2026), Oct 2026
62. Karya feature, TIME, 2023
63. Judge approves record $1.5 billion settlement involving Anthropic, Jurist, Jul 2026
64. Meta wins on fair use, for now, Authors Alliance, 26 Jun 2025
65. New York Times v. OpenAI case page, AI Lawsuit Tracker (secondary), 12 Oct 2026
66. Landmark ruling of the Munich Regional Court (GEMA v OpenAI), Bird & Bird, Nov 2025
67. Guidelines for providers of general-purpose AI models, European Commission, 28 Apr 2026
68. AI Office publishes final Code of Practice for general-purpose AI models, Covington, Global Policy Watch, Jul 2025
69. EDPB Opinion 28/2024: key takeaways, CMS, Dec 2024
70. Generative AI and GDPR enforcement in Europe, Cross-Border Data Forum (Christakis), 19 Mar 2026
71. DPC statement on Meta AI, Data Protection Commission Ireland (regulator), 21 May 2025
72. DPC launches investigation into X's data usage for AI training, William Fry, 15 Apr 2025
73. TDM and training AI models: UK government report and impact assessment, DLA Piper, 19 Mar 2026
74. Getty Images v Stability AI: permission to appeal, Burges Salmon, 19 Dec 2025
75. Interim Measures for the Management of Generative AI Services, China Law Translate, 10 Jul 2023
76. TC260 adopts security standard for generative AI services, Digital Policy Alert, 25 Apr 2025
77. General Understanding on AI and Copyright in Japan, APAA, Oct 2024
78. How does Singapore law treat the use of copyright works for AI training, IPOS (government), 2026
79. DPIIT closes consultation on working paper on generative AI and copyright, Digital Policy Alert, Jan 2026
80. DPIIT's attempt at bridging AI training and copyright, Takshashila Institution, 11 Dec 2025
81. Brazil AI bill PL 2338/2023: status, casrai.org (unverified site; check camara.leg.br), 25 Sep 2026
82. Brazilian Senate approves AI bill with copyright provisions, CISAC (creators' confederation), 12 Dec 2024
83. Constitutional Court ruling on the Copyright Amendment Bill, GoLegal, 21 Jul 2026