ArcFace Angular Margin Loss

ArcFace trains face embeddings on the surface of a sphere and demands an angular safety gap between identities, which sharpens open-set matching.

A face 20 degrees from its class center pays the 0.5-radian margin and still scores 42.3 against a rival 32.0, winning by 10.3 only by huddling close

Why Does This Exist?

Plain softmax separates training identities but leaves their embedding clusters sprawling: the boundary sits wherever is convenient, and unseen people at test time land in the gaps. Open-set recognition needs compact clusters with enforced gaps, not just separable ones.

ArcFace carves those gaps during ordinary classification training, with no triplet mining required. It keeps the softmax machinery and adds one geometric demand: each identity must win by an angular margin. This page covers the margin mechanism; the pipeline around it is face detection and recognition, the matching task it serves is face verification, and the mining-based alternative is triplet loss.

The one idea you need first. A face embedding is a vector direction: two photos of one person should point nearly the same way, and strangers should point apart. ArcFace L2-normalizes embeddings and class weights, so every logit is a cosine of an angle on a hypersphere, scaled by a factor $s$ .

Think of It Like This

Cones of personal space

Picture guests standing on a globe, each claiming a cone of personal space around their direction from the center. Softmax only asks that everyone stand somewhere inside their own cone. ArcFace widens every cone by a fixed angle, so guests must huddle near their cone's center line to fit. Tighter huddles with wider gaps between cones mean a newcomer is obviously inside one cone or none.

Where it stops: cones are re-carved every batch by gradient descent, not by a bouncer. Nobody assigns the space up front; the margin penalty teaches the embeddings to arrange themselves.

How It Actually Works

After normalization, the logit for the true class is $s \cdot \cos(\theta)$ , where $\theta$ is the angle between the face and its class direction and $s$ , often 64, controls how sharply angles convert into loss. ArcFace adds the margin $m$ to the target angle only, replacing $\cos(\theta)$ with $\cos(\theta + m)$ before the softmax. Rivals keep their plain cosines. The network must therefore pull each face much closer to its class center to reach the same logit, which compresses every identity cluster and widens every gap at once.

Worked example: the margin bill comes due

Take scale $s = 64$ and margin $m = 0.5$ radians, about 28.6 degrees. A face sits 20 degrees from its true class center. Without a margin its logit would be $64 \times \cos(20^\circ) \approx 64 \times 0.940 = 60.1$ . With the margin it pays the bill: $\theta + m = 48.6^\circ$ , so the logit is $64 \times \cos(48.6^\circ) \approx 64 \times 0.661 = 42.3$ .

The nearest rival sits 60 degrees away with logit $64 \times 0.5 = 32.0$ . The true class still wins by $42.3 - 32.0 = 10.3$ , but only because the face huddles close to its center. Without the margin the same face would win by $60.1 - 32.0 = 28.1$ and learn nothing about compactness. That missing comfort is the entire training signal.

Code

The point of this snippet is the margin bill: the same face scores far less once the target angle grows, which is what forces compact clusters.

1import numpy as np2
3s: float = 64.0  # logit scale4m: float = 0.5  # angular margin in radians, about 28.6 degrees5
6true_logit: float = s * float(np.cos(np.deg2rad(20.0) + m))  # face 20 deg out, margin paid7rival_logit: float = s * float(np.cos(np.deg2rad(60.0)))  # nearest rival, no margin8print(round(true_logit, 1), round(rival_logit, 1))  # -> 42.3 32.09print(round(true_logit - rival_logit, 1))  # -> 10.3

Drop the + m and the true logit returns to about 60.1, winning by a lazy 28.1: comfortable, and useless for open-set matching.

Watch Out For

A margin so large that training collapses

Pushing $m$ past about 0.5 radians on noisy labels makes the target logit nearly unreachable, and accuracy collapses instead of improving. The symptom is training loss that plateaus high while plain softmax on the same data trains fine.

Fix it by starting near 0.35, cleaning label noise first, and only then raising the margin toward 0.5. The margin amplifies whatever label quality you have, including the bad labels.

Normalizing only one side of the dot product

The margin math assumes logits are pure cosines, which needs both embeddings and class weights normalized. Normalize only the embeddings and weight magnitudes still leak into the scores, so the margin penalty lands unevenly. The symptom is a margin that barely changes accuracy no matter how you tune it.

Fix it by normalizing both sides and rescaling by $s$ afterward, exactly as the reference implementations do. Check the weight norms first whenever ArcFace seems to do nothing.

The Quick Version

ArcFace normalizes embeddings and class weights so every logit is a scaled cosine on a hypersphere.
Adding margin $m$ to the target angle forces compact, well-separated identity clusters.
The worked face pays 0.5 radians and scores 42.3 against a rival 32.0, winning by 10.3 instead of a lazy 28.1.
It trains with ordinary class labels, avoiding the triplet mining that FaceNet-style losses need.
Scale $s$ , often 64, controls how sharply angles convert into loss.
Too large a margin on noisy data collapses training, and skipping normalization on either side silences the margin.