Model Merging

Why Does This Exist?

You have a Llama 3 model that was fine-tuned heavily on Python code. Another team has a Llama 3 model fine-tuned heavily on French translation. You want a model that can translate French Python documentation.

You could gather both datasets and run a brand new, expensive full fine-tuning job. But neural networks have a fascinating mathematical property. Because both models started from the exact same pre-trained base model, their weights occupy the same dimensional space. You can calculate the difference between the base model and the Python model (the "task vector"), and simply add that delta to the French model.

Model Merging is the act of combining two or more fine-tuned models via pure arithmetic—no GPUs, no backpropagation, no training data required.

Think of It Like This

Mixing recipes

Imagine the base model is a plain vanilla cake recipe.

• Baker A adds chocolate chips (Delta A).
• Baker B adds a strawberry glaze (Delta B).

To make a chocolate chip strawberry cake, you don't need to reinvent the recipe from scratch. You just take the original vanilla recipe, add the chocolate chips, and add the strawberry glaze. As long as the base recipe was exactly the same, the modifications can be stacked.

How It Actually Works

You cannot just take any two models (like Mistral and Llama) and merge them; their architectures and weight shapes are completely different. Merging only works when models share the exact same base ancestry.

1. Task Vectors

A task vector is simply: $W_{task} = W_{finetuned} - W_{base}$. It isolates the specific weight changes that represent the new capability. To merge two capabilities, you add both task vectors back to the base model: $W_{merged} = W_{base} + (W_{task1} + W_{task2})$.

2. SLERP (Spherical Linear Interpolation)

If you simply average weights together (Linear Interpolation), you often destroy the magnitude and geometric properties of the high-dimensional vectors, leading to brain-damaged models. SLERP calculates the angle between the two weight vectors and interpolates along the curve of a sphere. This preserves the direction and magnitude of the weights much better than simple averaging.

3. TIES-Merging

When you merge three or four models together, their weight updates often contradict each other (one model pushed a weight positive, another pushed it negative). This causes "interference," degrading performance. TIES (TrIm, Elect Sign, and Merge) fixes this by:

1. Trimming the smallest 80% of weight changes (treating them as noise).
2. For the remaining 20%, enforcing a "majority vote" on whether the weight should be positive or negative.
3. Averaging only the weights that agree with the majority.

Watch Out For

The Frankenstein limit

Model merging is not magic. If you merge a coding model and a creative writing model, you often get a model that is slightly worse at coding and slightly worse at writing than the originals. While methods like SLERP and TIES mitigate interference, combining completely orthogonal tasks via arithmetic will always cause some capability degradation compared to training on a mixed dataset.

The Quick Version

• Model merging combines the capabilities of different fine-tuned models using pure math, requiring zero training compute.
• It only works if the models share the exact same base model architecture and starting weights.
• Simple averaging damages the model; advanced algorithms like SLERP (geometry preservation) and TIES (resolving interference) are used instead.
• It is incredibly popular in the open-source community for creating "Frankenstein" models that excel at multiple niche tasks.

What to Read Next

• Data Mixing and Curriculum is the traditional, expensive way to combine capabilities by training on a combined dataset.