Core ML

Mean Shift

Every point drifts uphill toward wherever the local density peaks, and where it stops is its cluster's mode. No k, no centroid, just the data's own shape.

Every point climbs toward the nearest peak of the local density, and points that converge on the same peak become one cluster, so the count of clusters falls out of the data instead of being chosen in advance

Why Does This Exist?

K-means needs $k$ told to it before it starts, and if the real number of groups in your data doesn't match, it will confidently produce the wrong one anyway — split one real cluster in two, or fuse two real clusters into one, because the objective it's minimizing has no way to say "that's not actually how many groups there are".

Here's the case we'll carry down the page. You're clustering customer sessions by two behavioral scores, and the true structure is three groups of very different sizes: a large cluster of routine browsers, a mid-sized cluster of frequent buyers, and a small, tight cluster of a specific niche segment. Nobody told you it was three. Ask k-means for two and it merges the small niche group into whichever larger group sits closer, and the merge looks completely clean on the plot — nothing about the output flags that a real group just vanished.

Mean shift never asks for $k$ . It asks a different question entirely: at every point, which way is uphill in density, and where does that hill actually peak?

Think of It Like This

Marbles released on a bumpy table

Scatter marbles across a table with several bumps and dips molded into its surface, and release each one where it landed. Every marble rolls downhill from its own position — except here, "downhill" for a marble means toward wherever the crowd of nearby marbles is thickest, so it's rolling toward density rather than gravity. It keeps rolling until it settles at the top of whichever bump is nearest to where it started.

Marbles that settle on the same bump belong together — that bump is their cluster, and its peak is the answer mean shift reports for that group. A marble on a small, separate bump settles on its own small bump regardless of how large the neighboring one is; size difference between bumps doesn't merge them. Nobody had to say in advance how many bumps the table has. The table's own shape decides.

How It Actually Works

Climbing the density, one point at a time

Every data point sits somewhere on an implicit surface: the local density of points around it, estimated using the distance metric chosen to compare points. Mean shift moves each point repeatedly toward the weighted mean of every other point within a radius called the bandwidth — literally the mean of its current neighborhood — then recomputes that neighborhood from the new position and repeats. Each step is a small climb toward denser ground.

The move always increases the density estimate at the point's location, the same guarantee that makes k-means's sum of squares fall every iteration — a quantity that only rises, bounded above, has to stop rising. So every point converges to a mode: a local peak where surrounding density stops increasing in every direction. Points converging to the same mode, or to modes close enough to merge, are labeled one cluster. Cluster count is whatever number of distinct modes the density surface happens to have — an output, not an input.

The one knob, and what it actually controls

Bandwidth sets the radius of the neighborhood used at every step, and it plays the same role $\varepsilon$ plays for DBSCAN: too small and the density surface is jagged with a peak wherever two points happen to sit close, producing far more clusters than are real; too large and distinct nearby bumps smooth into one shared peak, merging clusters that should stay separate. Estimating it from the data — commonly a quantile of pairwise distances — gives a reasonable starting point, but it is still the one lever that decides how many clusters come out, playing the role $k$ plays for k-means without ever being labeled that plainly.

What it buys over k-means, and what it costs

Uneven cluster sizes are handled the way the marbles handled them: a small, dense bump stays its own peak regardless of how large its neighbor is, because "uphill" is a local direction, not a comparison against every other cluster's size. That is exactly the failure mode k-means has no defense against. The cost is compute: every point's neighborhood has to be recomputed at every step, which scales quadratically with the number of points in the naive implementation, and mean shift never scales to the sizes k-means or a mini-batch variant reaches comfortably.

Show Me the Code

Three groups, deliberately uneven in size, and k-means told a wrong $k$ on purpose.

1import numpy as np2from sklearn.cluster import KMeans, MeanShift, estimate_bandwidth3
4rng = np.random.default_rng(5)5X = np.vstack([6    rng.normal([0.0, 0.0], 0.4, (150, 2)),7    rng.normal([5.0, 0.0], 0.4, (60, 2)),8    rng.normal([2.5, 4.0], 0.4, (30, 2)),9])  # three real groups, deliberately uneven in size10
11bandwidth = estimate_bandwidth(X, quantile=0.2)12modes = MeanShift(bandwidth=bandwidth).fit(X)13km_wrong = KMeans(n_clusters=2, n_init=10, random_state=0).fit(X)  # told the wrong k on purpose14
15print(f"mean shift found {len(np.unique(modes.labels_))} clusters on its own, bandwidth={bandwidth:.2f}")16print(f"mean shift cluster sizes: {np.bincount(modes.labels_)}")17print(f"k-means told k=2 merges two real groups, inertia={km_wrong.inertia_:.1f}")18# -> mean shift found 3 clusters on its own, bandwidth=1.0819# -> mean shift cluster sizes: [150  60  30]20# -> k-means told k=2 merges two real groups, inertia=523.1

Mean shift recovers all three groups, sizes and all, without ever being told there were three. K-means, forced to guess two, has no way to signal that its answer merged something real — the inertia number looks perfectly ordinary either way.

Watch Out For

Treating bandwidth as a free default

estimate_bandwidth's output is a reasonable starting point, not a validated answer, and small changes to it can swing the cluster count sharply — the same instability DBSCAN shows against its radius. A bandwidth that returns one giant cluster or as many clusters as data points is a sign the setting is wrong, not a finding about the data. Sweep a few values and check that the cluster count is stable across a reasonable range before trusting one run.

Running it on a dataset large enough that it never finishes

The naive algorithm recomputes each point's neighborhood at every iteration, which costs roughly the square of the point count per pass, and a dataset that k-means clusters in seconds can make mean shift impractical without a nearest-neighbor index accelerating the neighborhood lookups. Check the point count before committing to it in a pipeline that has to run repeatedly; past a few tens of thousands of rows, an accelerated implementation or a different method is usually the right call.

The Quick Version

Every point climbs toward the nearest peak of the local density estimate; points converging on the same peak form one cluster.
The number of clusters is an output, not an input — it falls out of how many distinct density peaks the data actually has.
Bandwidth is the one parameter, and it plays the same role $k$ plays for k-means and $\varepsilon$ plays for DBSCAN: too small over-clusters, too large merges real groups.
Uneven cluster sizes are handled naturally, since climbing is a local decision rather than a comparison against every other cluster.
Cost scales roughly quadratically with data size in the naive form, which limits it on large datasets without acceleration.