Black October, explainedAppsGuidesAI DictionarySmall FixesOn RecordAboutFollow the lab

AI accelerator

Also called ASIC, AI chip

An AI accelerator is a chip built to do the maths of AI much faster, and with less power, than an ordinary processor.

Chips and scaleCurious

Products, laws and numbers here change fast. Checked October 2026.

In plain words

Every computer has a CPU, a general chip that can do almost anything, a few steps at a time. AI needs one kind of work above all: multiplying huge grids of numbers. A chip built mainly for that work can do it many times faster and use less electricity. That is an AI accelerator.

The best known is the GPU, first made for video games and now the workhorse of AI. Others are designed only for AI from the start: Google's TPU, Amazon's Trainium and Microsoft's Maia in data centers, and the NPU in your phone that recognises your face.

The trade-off is flexibility. The more a chip is shaped around one job, the faster and cheaper it does that job, and the worse it is at everything else.

Everyday example

The juice stall's sugarcane press

At a juice stall, a good knife can peel, chop and slice anything, but slowly. The sugarcane press does exactly one thing, and does it in seconds. The vendor keeps both: the knife for odd jobs, the press for the queue. A computer works the same way. The CPU is the knife. An AI accelerator is the press, built around the one task AI repeats billions of times.

Where the example breaks: A sugarcane press cannot be reprogrammed. Most AI accelerators can run many different AI models, as long as the work is mostly the same kind of number-grid maths.

Go deeper

How it actually works

Accelerators win by giving up generality: more of the chip goes to multiply-and-add units and fast memory, less to the control logic a CPU needs for varied tasks.

  1. One operation, done massively

    Most AI work is matrix multiplication: rows of numbers times columns, then added up. Accelerators pack thousands of multiply-add units side by side. Google's first TPU had 65,536 of them in a single grid.

  2. Smaller numbers

    AI tolerates rough arithmetic. Accelerators often use 16-bit, 8-bit or even 4-bit numbers instead of the 32 or 64 bits much science needs, so each sum takes less silicon, memory and energy. See quantization.

  3. Feeding the maths

    Fast maths is useless if data arrives slowly. Data center accelerators sit beside stacks of high bandwidth memory. Cerebras goes further: its WSE-3 is one chip the size of a whole silicon wafer, with 44 GB of memory on the chip itself.

  4. Training chips and inference chips

    Some are built for training, which needs huge clusters of chips working together. Others focus on inference, answering users quickly and cheaply. Microsoft's Maia 200, announced in January 2026, is aimed at inference.

  5. Why cloud companies make their own

    Big cloud companies design chips like TPU, Trainium and Maia to cut costs and depend less on one supplier. They mostly use them in their own data centers and rent them out through their clouds, rather than selling the chips.

Main kinds
GPUs (flexible), custom AI chips called ASICs (built for AI only), and NPUs (small, inside phones and laptops)
ASIC means
Application-specific integrated circuit: a chip designed for one kind of task
Early custom AI chip
Google's TPU, in its data centers from 2015 and announced in May 2016
An unusual design
Cerebras WSE-3 (March 2024): one wafer-sized chip with 4 trillion transistors and 900,000 cores
As of October 2026
Custom chips in service include Google's Ironwood TPU, AWS Trainium3 (generally available from December 2025) and Microsoft's Maia 200, alongside GPUs from Nvidia and AMD

Software matters as much as silicon: a chip is only useful if AI tools run well on it, which is why CUDA gives Nvidia GPUs such a head start.

Questions people ask

What is the difference between a GPU and an AI accelerator?

A GPU is one kind of AI accelerator. It began as a graphics chip and is still flexible enough for graphics, science and AI. Other accelerators, such as Google's TPU, are built only for AI, which can make them cheaper to run for that work but less useful for anything else.

What is an ASIC in AI?

ASIC stands for application-specific integrated circuit, a chip designed for one kind of task. In AI it means custom chips like Google's TPU, Amazon's Trainium or Microsoft's Maia, laid out around the matrix maths that neural networks use, rather than general-purpose chips.

Is an NPU an AI accelerator?

Yes. An NPU, or neural processing unit, is a small AI accelerator built into phone and laptop chips. It runs features like face recognition, voice typing and on-device AI models using little battery, without sending your data to a data center.

Why are tech companies making their own AI chips?

Mainly cost and control. Companies that run AI for millions of users spend huge sums on chips and electricity. A chip shaped around their own workloads can cut both, and it reduces dependence on a single supplier. Designing one still takes years and large teams.

Do I have an AI accelerator in my phone?

Most likely, if your phone is recent. Apple has built a Neural Engine into iPhone chips since 2017, and Huawei and others add NPUs to theirs. Laptops sold as Copilot+ PCs since 2024 must have an NPU rated at 40 trillion operations per second or more.

Further reading

Prepared by Mundhra Labs. Reviewed by Aditya Mundhra. Checked October 2026.