In plain words
An AI model learns by studying examples, the way a student learns from past exam papers. The full set of examples is its training data.
For a spam filter, that means emails already marked spam or not spam. For a photo app, millions of labelled pictures. For a chatbot, a huge library of text: web pages, books, articles and code.
The data decides what the model can do. Train a model only on photos taken in daylight and it will struggle at night. Feed it text full of stereotypes and it can repeat them. People in the field sum this up as "garbage in, garbage out".
That is why AI companies spend so much effort collecting, cleaning and filtering data, and why questions about whose data was used, and with what permission, are now a big public debate.
Everyday example
Learning to cook from one notebook
Someone who learns to cook only from a Bengali grandmother's notebook will make wonderful fish curry, reach for mustard oil every time, and have no idea what to do with a tortilla. Their skill mirrors the notebook. Training data works the same way: a model is good at what its examples covered and weak or odd at whatever they left out.
Where the example breaks: A person can notice the gaps in what they learned and go and find more. A model cannot tell what its data was missing unless people test it and train it again.
How it actually works
Training data is gathered, cleaned, sometimes labelled, then split so the model can be tested fairly.
Labelled or not
Labelled data pairs each example with an answer (this photo is a cat) and is used in supervised learning. Unlabelled data, such as raw web text, is used in self-supervised learning, where the model sets its own task, like predicting the next word.
Where it comes from
Public web crawls, books and news, code, images with captions, examples labelled by people, and increasingly synthetic data produced by other models.
Cleaning and filtering
Raw data is full of duplicates, spam and harmful content. Teams filter it and often sample higher-quality sources more often than their size alone would suggest.
Training and test sets
Data is split. The model learns from the training set; a separate test set it never saw checks whether it truly learned or just memorised.
Scale for language models
GPT-3 (2020) trained on 300 billion tokens, mostly filtered web pages. Meta's Llama 3.1 (2024) used over 15 trillion tokens.
- GPT-3's mix
- 60% filtered Common Crawl web pages, 22% WebText2, 16% books, 3% Wikipedia (by sampling weight)
- Common Crawl
- A free archive of web pages collected since 2008, measured in petabytes
- ImageNet
- Millions of photos sorted into categories and labelled by people; first paper 2009
Questions people ask
What is training data used for?
It is what a model learns from. During training, the model makes predictions on the examples, measures its mistakes and adjusts its numbers. The patterns in the data become the patterns in the model.
Where does ChatGPT's training data come from?
OpenAI does not publish a full list for its recent models. The GPT-3 paper (2020) did: about 60% filtered web pages from Common Crawl, plus a curated web collection, two book collections and English Wikipedia. OpenAI has not published a breakdown like that for later models.
What is the difference between training data and test data?
Training data is what the model learns from. Test data is held back and used only at the end, to check performance on examples the model never saw. A model that scores well on training data but badly on test data has memorised rather than learned.
Can training data make AI biased?
Yes. If some groups, languages or situations are missing or misrepresented in the data, the model can work worse for them or repeat stereotypes. Fixing it means better data, careful testing and adjustments after training. See AI bias.
Is it legal to train AI on copyrighted data?
It depends on the country and is still being argued in courts, as of October 2026. Some argue training is a permitted use; rights holders argue it needs permission and payment. See AI and copyright.