In plain words
Normally, to train a model you gather all the data on one company's computers. But some data is too private to collect: what you type on your phone, patients' scans, bank records.
Federated learning flips it around. The current model is sent out to many devices. Each device improves its copy a little using only the data already on it, then sends back just the changes, a short summary of what it learned. The server averages the changes from many devices into a better shared model and sends that out again. The raw data never leaves the device.
Google introduced the name in 2016 and tested it in 2017 to improve suggestions in its phone keyboard. The same idea lets hospitals or banks build a model together without handing each other their records.
Everyday example
A recipe improved by many kitchens
A cooking club wants the perfect dal recipe, but nobody will share their family secrets. So the club sends one recipe to every member. Each cooks it at home, tweaks it based on their own family's taste, and sends back only notes like "a bit less salt, more jeera". The club combines everyone's notes into a new version and sends it out again. The secrets stay in each kitchen.
Where the example breaks: A note like "less salt" reveals little. A model update is a long list of numbers, and researchers have shown that updates can sometimes leak details of the data behind them, so real systems add extra protections.
How it actually works
McMahan and colleagues at Google described an approach that "leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates", and named it federated learning.
Send out the model
A central server picks a set of devices, often ones that are idle, charging and on Wi-Fi, and sends each the current model.
Train locally
Each device runs a few steps of gradient descent on its own data, improving its copy of the model's parameters.
Send back updates, then average
Devices return only their changes. The server averages them to produce the new shared model. Repeat for many rounds.
Protect the updates
Updates can leak information: Zhu and colleagues (2019) reconstructed training data from shared gradients. Defences include secure aggregation, which Google describes as letting the server read only the combined update once hundreds or thousands of users have taken part.
Hard parts
Devices come and go, have slow connections, and hold very different data from one another, which makes training slower and trickier than in one data centre.
- Named
- In a 2016 paper by H. Brendan McMahan and colleagues at Google
- Early use
- Google tested it in 2017 in Gboard, its Android keyboard, for query suggestions
- Who takes part
- Many phones, or a few organisations such as hospitals or companies
- What travels
- Model updates, not raw data
Questions people ask
What is federated learning in simple terms?
A way to train an AI model on data spread across many phones or organisations without collecting that data. The model travels to the data, learns locally, and only the learned changes are sent back and combined.
Is federated learning completely private?
Not by itself. Raw data stays on the device, but updates can sometimes reveal details of the data. That is why systems add protections such as secure aggregation, so no single update is visible. It reduces privacy risk rather than removing it.
Where is federated learning used?
Google described testing it in 2017 to improve keyboard suggestions on Android phones. It is also studied for hospitals and banks that want to build models together without sharing records.
How is federated learning different from on-device AI?
On-device AI means a model runs on your phone. Federated learning is about training: many devices help improve a shared model while keeping their data. The two often go together.
Further reading
- Communication-Efficient Learning of Deep Networks from Decentralized Data (McMahan et al.)arXiv, 2016
- Federated Learning: Collaborative Machine Learning without Centralized Training DataGoogle Research, 2017
- Advances and Open Problems in Federated Learning (Kairouz et al.)arXiv, 2019
- Deep Leakage from Gradients (Zhu, Liu, Han)arXiv, 2019