Small language model (SLM): what it is, explained

In plain words

The best-known chatbots run on enormous models in data centres. Each question travels over the internet to rooms full of powerful chips, and the answer comes back.

A small language model, or SLM, does the same kind of job with far fewer parameters, the numbers a model learns. Many have a few billion. Microsoft's Phi-3-mini, for example, has 3.8 billion; DeepSeek-V3 has 671 billion.

Being small means it can run on the device in our hand. That makes it fast, cheap to use, able to work offline and more private, because our words need not leave the phone. Apple, for example, runs a model of about 3 billion parameters on its devices for Apple Intelligence features.

The trade-off: small models know less and handle long, tricky reasoning less well. They shine on focused tasks.

Everyday example

The corner kirana versus the hypermarket

A hypermarket stocks everything, but it is a drive away and the queues are long. The kirana at the corner has far fewer items, yet it has what we need most days, it is right there, and it knows our usual order. A small language model is the kirana: less on the shelves, but close, quick and enough for everyday needs.

Where the example breaks: A kirana can order in anything it lacks. A small model cannot suddenly know more than it learned; for hard or unusual questions, apps hand over to a bigger model in the cloud.

How it actually works

There is no official size cut-off. As of October 2026, "small" usually means a few billion parameters, up to roughly ten or so: small enough for a phone, laptop or single graphics card.

Better data, not just less model

Microsoft's Phi models showed that training on carefully filtered and synthetic text lets a small model punch above its size. Phi-3-mini (April 2024) rivalled the far larger GPT-3.5 on standard tests, its makers reported.

Learning from big models

Many SLMs are trained with distillation: a big teacher model's answers guide the small student. Google trained its Gemma 2 2B and 9B models this way, and DeepSeek released small models distilled from its R1 reasoning model.

Squeezed to fit

Quantization stores the model's numbers with fewer bits so it fits in a phone's memory. Apple compressed its roughly 3 billion parameter on-device model to 2 bits per weight in 2025.

Where they run

Phones and laptops with dedicated AI chips (NPUs), cars, factory equipment, or cheap cloud servers. This is on-device AI. Many apps mix the two: a small model on the phone for quick jobs, a large one in the cloud for hard ones.

Phi-3-mini

Microsoft, April 2024: 3.8 billion parameters, trained on 3.3 trillion tokens, "small enough to be deployed on a phone"

Apple Intelligence

On-device model of about 3 billion parameters; opened to app developers in June 2025

Gemma 4

Google's open family, as of October 2026, includes E2B and E4B versions designed for phones

For scale

DeepSeek-V3 (December 2024): 671 billion parameters, about 180 times Phi-3-mini

Questions people ask

What is the difference between a small and a large language model?

Mostly size: billions of parameters versus hundreds of billions or more. Both work the same way, by predicting the next token. Large models know more and reason better; small ones are faster, cheaper and can run on our own devices.

Can a small language model run on my phone?

Yes, many already do. Apple's on-device model, for example, has about 3 billion parameters and runs on its phones. Enthusiasts also run open small models such as Gemma, Phi or Llama variants on their phones with free apps.

Are small language models private?

They can be. If the model runs fully on the device, our text does not need to be sent anywhere. But apps may still send some requests to the cloud, so it is worth checking each app's privacy settings.

Why use a small model if large ones are better?

Cost, speed and privacy. For a focused job, such as sorting emails or pulling details from forms, a small model is often good enough at a tiny fraction of the cost, and it can work without internet.