In plain words
Point a phone camera at someone doing yoga and a pose estimation model marks dots on their nose, shoulders, elbows, wrists, hips, knees and ankles. Join the dots and you get a stick figure that moves as they move.
That stick figure is a few numbers instead of millions of pixels: where each joint is. From those numbers a computer can measure things, like how far your knee bends, whether your back is straight, or whether someone is standing, sitting or lying down.
It is part of computer vision. It works out how a body is positioned, not who the person is. Telling people apart by face is a separate job, facial recognition.
Everyday example
A yoga teacher watching from the side
In a class, the teacher does not study your clothes or face. Their eyes go to your joints: is the front knee over the ankle, are the hips level, are the arms in line with the shoulders? They picture the lines between those points and correct you. A fitness app does the same with dots and lines on the video.
Where the example breaks: A teacher knows anatomy and can see you are tired. The model only places dots where joints usually appear in its training photos, so baggy clothes, odd angles or a hidden arm can put a dot in the wrong place with full confidence.
How it actually works
Pose estimation turns each body into a set of keypoints: labelled positions in the image. A common standard, from the COCO dataset, uses 17 keypoints per person.
Learn from labelled photos
People mark the joints by hand on tens of thousands of photos. The COCO keypoints set alone has over 150,000 people labelled with 17 points each. A neural network learns to place the same points on new images. This is supervised learning.
Predict a heat map for each joint
Many models output one map per joint, glowing where that joint probably is. The brightest spot becomes the dot. Early deep-learning work in 2013, called DeepPose, predicted the joint coordinates directly instead.
Top-down: find people, then joints
One approach first draws a box around each person, then finds that person's joints. It is accurate, but gets slower with every extra person, and fails if the person detector misses someone in a crowd.
Bottom-up: find all joints, then group them
The other finds every elbow and knee in the picture first, then works out which belong together. OpenPose (2016) learned the direction of each limb to join the right dots, so its speed barely depends on how many people are in view.
Go 3D and follow over time
Some models also estimate depth, giving a 3D skeleton from an ordinary camera. On video, the points are tracked frame to frame (see object tracking) so the figure moves smoothly.
- Early deep-learning method
- DeepPose, by Alexander Toshev and Christian Szegedy, December 2013
- Multi-person milestone
- The Carnegie Mellon method behind OpenPose won the first COCO keypoints challenge in 2016
- Common keypoint counts
- 17 in the COCO standard. Google's MediaPipe pose model tracks 33 points in 3D
- Where you meet it
- Fitness and yoga apps, posture checks, sports and dance analysis, characters in game engines, robots learning from people
Pose models can run on a phone in real time, which is why they show up in home workout apps and video calls.
Questions people ask
Is pose estimation the same as facial recognition?
No. Pose estimation finds where body parts are, so it can tell a squat from a lunge. Facial recognition tries to tell who someone is. A pose model's stick figure carries no name, though the video it came from may still be personal.
How do fitness apps count my reps?
One common way: the app runs pose estimation on each camera frame and sorts every frame into a position, like "down" or "up" in a push-up. Each time the model becomes confident you have gone down and then come back up, it counts one rep.
Can pose estimation run on a phone?
Yes. Models such as Google's MediaPipe pose landmarker are built to run on the device, in real time, from an ordinary phone or laptop camera. No special suit or extra cameras are needed.
What are keypoints in pose estimation?
Keypoints are the labelled positions the model finds, such as left wrist or right knee. Each has an x and y position in the image, often a confidence score, and sometimes a depth value.
Was this clear?
1 is confusing, 5 is crystal clear. One tap, no account needed.
Spotted something wrong, or know a better example? Suggest a change