Deep learning: what it is, explained simply

In plain words

Show a computer a photo and all it gets is a grid of numbers, one per pixel. Getting from those numbers to "a dog on a sofa" is a long way.

Deep learning breaks that journey into many small steps. A neural network is arranged in layers. Early layers pick up simple things like edges. The next combine edges into shapes. Later layers combine shapes into eyes, ears or wheels, and finally whole objects.

Nobody tells the layers what to look for. During training, each layer learns whatever features help the final answer. "Deep" just means many layers, not deep understanding.

This approach is behind face recognition on phones, voice typing, translation apps and every modern chatbot.

Everyday example

Reading a doctor's handwriting

When a chemist reads a scrawled prescription, it happens in stages. First strokes, then letters, then whole words, then the medicine name that fits. Each stage uses the one before. A deep network works the same way, layer by layer, except it learned its own stages from millions of examples instead of years behind a counter.

Where the example breaks: The chemist knows what medicines do. A network's layers are not neatly "letters" and "words": they are learned patterns that often match no human idea, and the network has no sense of what its answer means.

How it actually works

Deep learning is a branch of machine learning. It worked once depth, data and computing power arrived together.

Layers of artificial neurons

Each layer is a set of simple units that take numbers in, weight them, add them up and pass the result through an activation function. Stacking dozens or hundreds of layers lets the network represent very complicated patterns.

Features are learned, not hand-made

Older systems needed people to design features, such as "count the edges". A deep network learns its own features at every level, from raw pixels or sound waves up to abstract concepts.

Backpropagation adjusts every layer

After each guess, the error is sent backwards through the network. Backpropagation works out how much each weight contributed, and gradient descent nudges them all.

Scale made it work

The ideas existed for decades. Large labelled datasets and fast GPUs arrived around 2010. In 2012 a deep network later known as AlexNet won the ImageNet contest with a top-5 error of about 15%, against about 26% for the next team.

Families of designs

Different layer designs suit different data: convolutional networks for images, recurrent networks for sequences, and transformers, which now power most language models and many image models.

Landmark review

"Deep learning" by Yann LeCun, Yoshua Bengio and Geoffrey Hinton, Nature, May 2015

Turning point

AlexNet, 2012: about 60 million parameters, trained on about 1.3 million images

Award

LeCun, Bengio and Hinton won the 2018 Turing Award, announced in March 2019

Used for

Image and speech recognition, translation, chatbots, image generation

Questions people ask

What is the difference between deep learning and machine learning?

Deep learning is a subset of machine learning. Classic machine learning often relies on features chosen by people and simpler models. Deep learning uses neural networks with many layers that learn their own features straight from raw data, given enough examples and computing power.

Why is it called deep learning?

Because the network has many layers stacked between input and output. Depth refers to that stack, not to deep understanding. Google's glossary calls a network with more than one hidden layer a deep model; modern ones can have hundreds of layers.

Is ChatGPT deep learning?

Yes. ChatGPT runs on a large language model, which is a very large deep neural network built on the transformer design, with billions of parameters arranged in many layers.

Who invented deep learning?

It grew over decades from the work of many researchers. Geoffrey Hinton, Yann LeCun and Yoshua Bengio are often called its pioneers and shared the 2018 Turing Award for making deep neural networks "a critical component of computing".

Why does deep learning need so much data?

A deep network has millions or billions of adjustable parameters. To set them well without simply memorising, it needs many varied examples. With too little data it tends to overfit: great on its training examples, poor on new ones.