In plain words
An AI model is, underneath, a huge set of numbers called parameters. Training sets those numbers. Fine-tuning nudges them to teach a new skill.
Now imagine two copies of the same starting model. One was fine-tuned to be good at maths, the other to write good Hindi. Model merging combines their numbers, one by one, for example by averaging them, to make one model that keeps much of both skills.
No new training is needed, so merging is cheap. It can even run on an ordinary computer without a powerful graphics chip. People sharing open-weight models online have published thousands of merged models.
The catch: it only works well when the models share the same starting point. Two unrelated models cannot simply be averaged.
Everyday example
Two cooks adjusting the same family recipe
Two cousins start from grandma's dal recipe. One tweaks it for more spice, the other for a richer texture. Each change is small. Because they began from the same recipe, you can apply both sets of tweaks to one pot and keep most of each improvement. Mixing grandma's dal with a stranger's unrelated recipe would just make a mess.
Where the example breaks: Recipe tweaks are a handful of clear choices. A model has billions of numbers, and tweaks can clash: one model pushed a number up, the other pushed it down. Good merge methods exist mainly to handle those clashes.
How it actually works
Merging works directly on the parameters. Most methods start by asking what each fine-tune changed compared with the shared base model.
Average the weights
The simplest merge is a weighted average of matching numbers. A 2022 study called it a "model soup": averaging several fine-tunes of one image model beat the best single one, at no extra cost when running it.
Work with task vectors
Subtract the base model's numbers from a fine-tuned model's numbers and you get a task vector: the change that added a skill. Add several task vectors to the base to combine skills. Subtract one to weaken a behaviour.
Resolve clashes
Methods like TIES-merging drop tiny changes, then for each number pick the direction most models agree on, and average only the values that agree. This cuts interference between skills.
Search for a good recipe
Which models, which method and what mix? Tools like the open-source MergeKit make trying recipes easy, and some teams use automated search to find good merges. Every merge still needs testing, because results can surprise.
- Model soups
- 2022 paper: averaged fine-tunes of one image model reached 90.94% top-1 accuracy on ImageNet, beating the best single fine-tune
- Task arithmetic
- 2022: showed skills can be added or removed by adding or subtracting weight differences
- Hardware
- MergeKit says merges can run entirely on a CPU, or with as little as 8 GB of GPU memory
- Not the same as
- Mixture of experts, which keeps separate expert parts inside one model and picks between them while running
Questions people ask
Why do people merge AI models?
To combine skills cheaply. Training a model costs a lot of computing. Merging two fine-tunes needs no training and can run on modest hardware, giving one model that handles both jobs, without keeping two models running.
Can you merge any two models?
Not usefully. The common methods assume the models are fine-tunes of the same base model, with the same design and size. Their numbers then line up meaningfully. Models trained separately from scratch have numbers that do not correspond.
Is a merged model better than the originals?
Sometimes, often not on every task. A merge can keep most of each skill, and averaging can even improve accuracy, but some ability is usually lost where skills clash. That is why merged models are tested on benchmarks before use.
What is the difference between merging and distillation?
Merging combines the numbers directly, with no training. Distillation trains a new, usually smaller model to copy the outputs of a bigger one. Merging is faster. Distillation can change the model's size and design.
Was this clear?
1 is confusing, 5 is crystal clear. One tap, no account needed.
Spotted something wrong, or know a better example? Suggest a change
Further reading
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (Wortsman et al.)arXiv, 2022
- Editing Models with Task Arithmetic (Ilharco et al.)arXiv, 2022
- TIES-Merging: Resolving Interference When Merging Models (Yadav et al.)arXiv, 2023
- Arcee's MergeKit: A Toolkit for Merging Large Language Models (Goddard et al.)arXiv, 2024