Latent space in AI: what it is, explained simply

In plain words

A photo on your phone is millions of numbers, one for every tiny dot of colour. Most of them are predictable: if one pixel of sky is blue, its neighbour probably is too. A model can learn to squeeze the picture into a much shorter list of numbers that keeps only what matters: the shape of the face, the light, the pose.

That short list is a point in the model's latent space. "Latent" means hidden: you never see these numbers, but the model works with them. Nearby points describe similar things, and moving in one direction can change one quality, like turning a smile up or a head to the side.

This is why image generators can be fast. Stable Diffusion, for example, does its heavy work on the compact version and only turns it back into full pixels at the very end. It is the same idea as an embedding, used inside models that create things.

Everyday example

Describing a face to a sketch artist

A witness does not list every pore. They say: round face, thick eyebrows, small smile, glasses. A few notes, yet the artist can draw a recognisable face. Change "small smile" to "big smile" and only the mouth changes. Latent space is the model's own set of notes, learned rather than chosen, from which it can redraw the whole picture.

Where the example breaks: The witness's notes are words we understand. A model's latent numbers are not labelled. Some directions turn out to match things like smiling, but most mix many qualities in ways no person chose or can easily read.

How it actually works

Latent spaces are learned by models that must compress data and then rebuild it.

Encoder and decoder

An autoencoder has two halves: an encoder squeezes the input into a small set of numbers, and a decoder rebuilds the input from them. Training rewards accurate rebuilding, so the small set is forced to keep what matters. That small set is the latent representation.

Making the space smooth

A variational autoencoder (VAE), introduced in 2013, encodes each input as a small cloud rather than a single point. This makes the space smooth: points between two known images also decode to sensible images.

Directions mean something

Researchers found that simple arithmetic works in many latent spaces. In a 2015 paper on convolutional GANs, averaging the codes for left-facing and right-facing faces gave a "turn" direction that could rotate new faces.

Doing the work in the small space

Latent diffusion models, the basis of Stable Diffusion, run their diffusion steps on the compressed version. A 512 by 512 colour image has 786,432 numbers. Its latent has 16,384, 48 times fewer, which makes generation far cheaper.

Word meaning

Latent: present but hidden. The numbers exist inside the model but are never shown

Key paper

"Auto-Encoding Variational Bayes", Kingma and Welling, December 2013

In practice

Stable Diffusion shrinks each side of an image by 8, from 512 by 512 pixels to a 64 by 64 latent with 4 channels

Close relatives

Embeddings and vectors: latent codes are vectors, and an embedding space is one kind of latent space

Real latent spaces have far more than two dimensions: Stable Diffusion's has 16,384. Any flat picture of one, including ours, is a simplification.

Questions people ask

What is the difference between latent space and embedding space?

They overlap a lot. Both turn data into lists of numbers where similar things sit close together. "Embedding" is used more for search and language, to compare meanings. "Latent space" is used more in generative models, where the point can also be turned back into an image or sound.

Why is it called latent?

Latent means hidden or not directly visible. The model's internal numbers are never shown to users, yet they capture the underlying qualities of the data, such as pose, colour or style.

What does it mean to move through latent space?

Picking points between two codes and decoding each one. With faces, walking from a frowning face's code to a smiling face's code produces a smooth series of faces that slowly change expression. This is called interpolation.

Is latent space used in ChatGPT?

In a sense. Every large language model turns tokens into internal vectors, and those vectors live in a learned space. People usually call those embeddings or hidden states. The term latent space is most common for image, audio and video models.