Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

Computer vision

Also called CV, machine vision

Computer vision is the part of AI that teaches computers to make sense of photos and video: what is in them, where, and what is happening.

In plain words

When you look at a photo, you see a cat on a mat. A computer sees a grid of tiny dots called pixels, and each pixel is just a few numbers for its colour. Nothing in those numbers says "cat".

Computer vision is the work of getting from those numbers to meaning. Instead of writing rules by hand, we show a model millions of labelled pictures and let it learn which patterns of numbers go with which things. First it picks up edges, then shapes, then whole objects.

That is what lets your phone sort photos by face, read a number plate at a car park gate, check crops for disease or help a doctor spot a problem on an X-ray. It sits under the wider umbrella of artificial intelligence and most of it today runs on deep learning.

Everyday example

Teaching a child to spot a mango

Nobody gives a child a rulebook for mangoes. They see hundreds: green ones, yellow ones, cut ones, one in a picture book. Slowly they learn what matters, the shape and the colour, and what does not, the plate or the light. Computer vision learns the same way, from many labelled examples, until it can name a mango it has never seen.

Where the example breaks: A child understands what a mango is: you can eat it, it grows on a tree. A vision model only links patterns of pixels to a label, so an odd angle, poor light or an unusual picture can fool it in ways that would never fool a person.

Go deeper

How it actually works

Most computer vision today is a trained neural network that turns a grid of pixel numbers into an answer.

  1. A picture becomes numbers

    A colour photo is stored as three grids, one each for red, green and blue. Each cell holds a brightness from 0 to 255. A 12 megapixel phone photo is about 36 million numbers.

  2. Layers find patterns

    A convolutional neural network slides small filters across the image to find edges and textures, then combines them into shapes and parts. Newer vision transformers cut the image into patches and use attention to relate them.

  3. Different jobs, different outputs

    The same building blocks power several tasks: image recognition (what is in the picture and where), segmentation (which pixels belong to which thing), OCR (reading text), facial recognition and tracking objects across video frames.

  4. It learns from labelled examples

    Models are trained on huge sets of labelled images. ImageNet, released in 2009, sorted millions of photos by category. Its yearly challenge, with 1,000 classes, became the standard test for a decade.

An early start
MIT's "Summer Vision Project" memo, July 1966, hoped students could get a computer to split a picture into regions over one summer
Turning point
2012: AlexNet, a deep network from the University of Toronto, won the ImageNet challenge with a 15.3% top-5 error, against 26.2% for the next entry
Common uses
Medical scans, crop monitoring, factory quality checks, self-driving cars, phone photo search
Also called
CV, or machine vision in factories

Vision is now one sense among several: Multimodal AI combines it with language and sound.

Questions people ask

What is computer vision used for in daily life?

Unlocking a phone with your face, searching your photo gallery for "beach", reading a number plate at a car park, scanning a document into text and pointing your camera at a sign to translate it. Hospitals, farms and factories use it too, often quietly in the background.

Is computer vision the same as image recognition?

No. Image recognition is one job inside computer vision: naming what is in a picture. Computer vision is the whole field, which also covers reading text, tracking motion in video, measuring depth and understanding scenes.

Is computer vision part of AI?

Yes. It is one of the oldest branches of artificial intelligence. Early systems used hand-written rules for edges and shapes. Since about 2012, nearly all of it uses machine learning, mostly deep neural networks.

How does a computer see an image?

As numbers. Each pixel holds values for red, green and blue light. A trained model reads those numbers in layers, finding edges first, then shapes, then objects, and finally outputs a label, a box or a description.

Can computer vision make mistakes?

Yes. It can be fooled by poor light, odd angles, or things it rarely saw in training. It can also be less accurate for some groups of people if its training photos were unbalanced, which is a form of AI bias.

Further reading