In plain words
You tap the mic on your keyboard, say "call Mum at six", and the words appear. Your phone has turned sound into text. That is speech recognition, also called speech-to-text.
Sound reaches the phone as a wave: air pressure going up and down thousands of times a second. The model learned, from many thousands of hours of recorded speech with matching transcripts, which wiggles in that wave go with which sounds, and which sounds make which words.
Sounds alone are often ambiguous. "Six" and "sticks" can sound close in a noisy auto. So the model also weighs which words make sense together, the same way you fill gaps on a bad phone line. "Call Mum at six" is far more likely than "call Mum at sticks". That mix of hearing and context is what makes captions, voice assistants and voice notes to text work.
Everyday example
Taking notes in a noisy class
A student copies down a lecture over a ceiling fan's hum. Some words come through clearly. For blurry ones, they guess from the sentence: the teacher said "photosynthesis", not "photo synthesis", because it is a biology class. Speech recognition does both too: it listens to the sounds, then picks the words that make most sense in context.
Where the example breaks: The student understands the lecture. The model only matches sounds and likely word patterns, so names, mixed languages, strong accents it rarely heard or unusual phrases can come out confidently wrong.
How it actually works
Modern systems are trained end to end, but they still do three jobs: hear, spell and judge.
Sound becomes a picture
The audio, tens of thousands of numbers per second, is cut into short slices, roughly 25 milliseconds each. Each slice is turned into a spectrogram column showing how much energy sits at each pitch.
An acoustic model listens
A neural network, often a transformer, reads the spectrogram and works out which speech sounds or word pieces are present over time.
A language model judges
Many candidate transcripts are possible. A language model part scores which word sequences are likely, so "at six" beats "at sticks". Older systems kept these parts separate; newer ones like OpenAI's Whisper learn them together.
Measured by word error rate
Accuracy is reported as word error rate: the share of words wrongly added, dropped or swapped. Background noise, accents and crosstalk all push it up.
- Early machine
- 1952: Bell Labs' "Audrey" recognised spoken digits from a single speaker
- A famous demo
- IBM's Shoebox (1961) understood the digits 0 to 9 and six command words
- Open model
- September 2022: OpenAI released Whisper, trained on 680,000 hours of web audio in many languages
- Not the same as
- Voice recognition, which works out who is speaking rather than what they said
The reverse job, turning text into a voice, is text-to-speech. Both are part of natural language processing.
Questions people ask
What is the difference between speech recognition and voice recognition?
Speech recognition works out what was said and writes it down. Voice recognition works out who is speaking, for example to check it is really you. The words sound similar, but the jobs are different.
How accurate is speech recognition?
In quiet conditions with a common accent and language, modern systems get the large majority of words right. Accuracy drops with background noise, several people talking at once, rare names, and accents or languages that were thin in the training data.
Does speech recognition work in Indian languages?
Yes, increasingly. Google added voice typing in eight more Indian languages back in 2017, and OpenAI's open Whisper model was trained on audio in close to 100 languages. Quality varies by language, and mixing languages mid-sentence, like Hinglish, is still harder.
Is my phone always listening?
Assistants wait in a standby mode, checking short snippets of audio on the device for a wake phrase such as "Hey Google". Google says nothing is sent to its servers until that phrase is detected. Some recognition now happens on the phone itself, which is On-device AI. You can turn wake words off in settings.
What is ASR?
ASR stands for automatic speech recognition. It is the technical name for the same thing: software that converts spoken audio into text.