In plain words
Point your phone camera at a plant and an app tells you it is a tulsi. Search your gallery for "dog" and every dog photo appears. That is image recognition: software that names the things in a picture.
It learns the way we learn faces in a new class: by seeing many examples. Shown thousands of photos labelled "dog", a model picks up what dogs tend to look like, so it can spot one it has never seen.
There are two common jobs. Classification gives one label for the whole picture: "this is a dog". Detection goes further and draws a box around each thing it finds, with a score for how sure it is. Both belong to computer vision, the wider field of machines making sense of what cameras see.
Everyday example
A sharp-eyed shopkeeper
A kirana shopkeeper glances at a crowded counter and instantly knows: two biscuit packets, one soap, a bottle of oil. Years of seeing the same items taught them. Image recognition works the same way. Trained on many labelled photos, it looks at a new one, names each item, points to where it sits and says how sure it is.
Where the example breaks: The shopkeeper knows what soap is for and would notice a strange new product. A model only matches patterns it has seen. Show it something outside its training, and it will confidently give the nearest label it knows, even when that is wrong.
How it actually works
Under the hood, image recognition is usually a trained neural network that maps pixels to labels and scores.
Learn from labelled photos
A model is trained on large sets of labelled images. For detection, each label also includes a box drawn by a person around the object. This is supervised learning.
Turn pixels into features
A convolutional network or a vision transformer reads the image in layers, from edges and textures up to parts and whole objects.
Score every label
For classification, the last layer gives a probability for each label the model knows, such as 1,000 classes in the ImageNet challenge. The highest score wins. This is a form of classification.
Find where things are
Detectors also predict boxes: where each object starts and ends, plus a score. Boxes below a confidence threshold are dropped, and overlapping boxes for the same object are merged.
- The big test
- ImageNet (2009) and its yearly challenge, which ran until 2017
- Breakthrough
- 2012: AlexNet cut the winning top-5 error to 15.3%, from 26.2% for the next entry
- Beyond the benchmark
- 2015: an ensemble of ResNet models reached 3.57% top-5 error on ImageNet
- Everyday scale
- Google said in June 2023 that Lens handles more than 12 billion visual searches a month
Questions people ask
What is the difference between image recognition and object detection?
Image recognition is the broad idea of naming what is in a picture. Image classification gives one label to the whole image. Object detection finds each object, draws a box around it and labels it. Detection is one kind of image recognition.
What are examples of image recognition?
Google Lens identifying a plant, a photo app grouping pictures by person, a car spotting pedestrians, a hospital tool flagging something on an X-ray, and a factory camera catching a faulty part on a moving belt.
How accurate is image recognition?
On standard tests, very accurate: top models make few mistakes on the ImageNet benchmark. In the real world, accuracy drops with poor light, odd angles, rare objects or photos unlike the training data. Scores are a guide, not a guarantee.
Is image recognition the same as facial recognition?
No. Image recognition names things, like "a face" or "a dog". Facial recognition goes further and works out whose face it is by comparing it with stored faces. That raises privacy questions that plain object labels do not.