In plain words
The main chip in a phone is really several engines on one piece of silicon. The CPU runs apps. The GPU draws the screen. And in most recent phones and many new laptops, there is a third: the NPU, or neural processing unit.
An NPU is built for one kind of work: the maths inside neural networks, which is mostly multiplying and adding long lists of numbers. Because it does nothing else, it can do that work using much less power than the CPU or GPU would.
That is what makes on-device AI practical. Unlocking your phone with your face, live captions and translation, or summarising a voice recording can all run on the NPU, without draining the battery or sending your data to a server.
Everyday example
A pressure cooker in the kitchen
A kitchen has a stove that can cook anything. But for dal and rice, most homes reach for the pressure cooker: it does that one job faster and with less gas. Nobody throws away the stove; each tool handles what it is best at. In a phone, the CPU is the stove and the NPU is the pressure cooker for AI.
Where the example breaks: A pressure cooker is a separate pot. An NPU is part of the same chip as the CPU and GPU and shares memory with them, and the phone's software decides which engine gets each job, without you choosing.
How it actually works
NPU is a general name; makers use their own, such as Apple's Neural Engine, or the TPU block inside Google's Pixel Tensor chips.
Built around multiply-add
Like a data center TPU, an NPU dedicates most of its area to arrays of multiply-add units, plus memory placed close to them so numbers do not travel far. Moving data costs more energy than the maths itself.
Low precision
NPUs work best with small numbers, such as 8-bit values or fewer instead of 32-bit decimals. Models are compressed to suit them, a process called quantization. Apple squeezes its on-device model to about 2 bits per weight.
Measured in TOPS
NPU speed is quoted in TOPS: trillions of operations per second. It is a peak figure at low precision, so it is useful for rough comparison but not a guarantee of real-world speed. See FLOPs.
The software decides
Apps do not usually talk to the NPU directly. They hand a model to the operating system's AI framework, which runs it on the NPU, GPU or CPU, whichever suits.
- Early phone NPUs
- September 2017: Huawei's Kirin 970 with a dedicated NPU, and Apple's A11 Bionic with its Neural Engine (600 billion operations per second, used for Face ID)
- A 2024 laptop chip
- Apple's M4 Neural Engine: up to 38 trillion operations per second
- Windows AI PCs
- Copilot+ PCs (May 2024) require an NPU of 40 trillion operations per second or more
- Huawei's 2017 claim
- Up to 50 times the energy efficiency of the phone's CPU on AI tasks
Questions people ask
What does an NPU do?
It runs AI models, such as face recognition, voice transcription, photo enhancement or a small language model, on your device. It does this maths faster and with much less power than the main processor, so AI features can run without draining the battery or needing the internet.
What is the difference between an NPU, a GPU and a CPU?
The CPU is the general manager that runs everything. The GPU does many calculations in parallel, mainly for graphics. The NPU is narrower still: built only for neural network maths, which makes it the most power-efficient of the three for AI.
Do you need an NPU in a laptop?
Not for most people today. AI chatbots run in the cloud, so any laptop can use them. An NPU helps with built-in features that run locally, such as live captions with translation, and Microsoft's Copilot+ PC features require one.
What does TOPS mean on an NPU?
Trillions of operations per second: how many basic maths steps the NPU can do at peak, usually with small 8-bit numbers. A higher number means more raw capacity, but real speed also depends on memory and software, so compare TOPS only roughly.