Clustering

Unsupervised clustering discovers latent structure in data without target labels. When observations arrive with coordinates alone, grouping relies purely on geometric proximity. The objective is to partition the points into cohesive clusters such that distances between points within the same cluster are minimized, while separation between different clusters is maximized.

Nearest Center Wins

K-Means establishes initial centroids and assigns each point to the closest center using Euclidean distance:

$C_k = \{ x_i : \| x_i - \mu_k \| \le \| x_i - \mu_j \|, \forall j \}$

Placing candidate centroids at $\mu_1 = 1.0$ and $\mu_2 = 2.0$ tests observation $x = 9.0$. The distance to the first center is $|9 - 1| = 8.0$, while the distance to the second center is $|9 - 2| = 7.0$. Because $7.0 < 8.0$, point $9$ assigns to center $2$. Cluster membership is governed strictly by comparative distance.

Assign, Then Move

Evaluating the full four-point fixture $\{1, 2, 9, 10\}$ against initial centers $\mu_1=1.0$ and $\mu_2=2.0$ establishes the Voronoi partition threshold at the midpoint:

$\tau = \frac{\mu_1 + \mu_2}{2} = \frac{1.0 + 2.0}{2} = 1.5$

Observation $1$ falls below $1.5$ and forms Cluster 1 alone. Observations $2$, $9$, and $10$ exceed $1.5$ and assign to Cluster 2. This lopsided $1$-versus-$3$ split is an expected consequence of arbitrary centroid initialization that the subsequent update step corrects.

Centers Chase Means

Once assignments are fixed, each centroid leaps directly to the empirical center of mass of its assigned members:

$\mu_k = \frac{1}{|C_k|} \sum_{x_i \in C_k} x_i$

Center $1$ has member $\{1\}$, remaining at $\mu_1 = 1.0$. Center $2$ averages members $\{2, 9, 10\}$:

$\mu_2 = \frac{2 + 9 + 10}{3} = \frac{21}{3} = 7.0$

Center $2$ jumps from $2.0$ to $7.0$ in a single recalculation, centering itself over its assigned data mass.

Error Collapses

In round 2, the midpoint moves to $(1.0 + 7.0)/2 = 4.0$, cleanly grouping $\{1, 2\}$ into Cluster 1 and $\{9, 10\}$ into Cluster 2. Recalculating group means settles $\mu_1 = 1.5$ and $\mu_2 = 9.5$. No observation changes assignment, signaling convergence. Total Within-Cluster Sum of Squares (WCSS) error collapses from an initial $113.0$ down to $1.0$:

$J = (1 - 1.5)^2 + (2 - 1.5)^2 + (9 - 9.5)^2 + (10 - 9.5)^2 = 4 \times 0.25 = 1.0$

Stop At Elbow

Selecting the number of clusters $K$ requires balancing model parsimony against residual variance. Plotting inertia across values of $K$ reveals the elbow bend:

• $K=1$: WCSS $= 65.0$
• $K=2$: WCSS $= 1.0$ (error drops by $-64.0$)
• $K=3$: WCSS $= 0.5$ (diminishing return of $-0.5$)

Adding the second centroid captures almost all structured variance. Beyond $K=2$, incremental clusters provide negligible benefit while adding unwarranted complexity.

Where It Breaks

Because K-Means optimizes distance to point centroids, it implicitly assumes convex, isotropic, spherical clusters separated by linear Voronoi hyperplanes. When presented with interleaved non-convex geometries—such as interlocking half-moon crescents—the linear boundary slices straight across both manifolds, erroneously combining opposite halves into arbitrary geometric hemispheres. Resolving non-linear continuous shapes requires density-based (DBSCAN) or spectral clustering algorithms.

The Quick Version

• Unsupervised clustering organizes unlabelled points from distance metrics alone.
• Point 9 joins center 2 because its distance of 7.0 beats 8.0.
• Voronoi midpoint partitioning initially splits points 1 against 3.
• The update step moves centroids to group means: c2 leaps from 2.0 to 7.0.
• Final settled centroids at 1.5 and 9.5 reduce total squared error from 113.0 to 1.0.
• The elbow plot bends sharply at K=2 (drop of -64.0 vs marginal -0.5 for K=3).
• Linear Voronoi boundaries bisect non-convex crescent manifolds, requiring density methods.