Visual explainer

Word Embeddings

How computers represent words as dense vectors, allowing semantic meaning to be treated as math.

One-hot puts cat as far from dog as from car.

You give each word its own slot with one 1 and thousands of zeros. Cat, dog, and car all sit equally far apart, so you'll see no hint of meaning.

Dense Beats Sparse

A dense vector of 100 numbers replaces 10,000 sparse slots.

You swap 10,000 sparse slots for 100 dense numbers. Cat becomes [2.0, 1.0] with every slot filled, so small shifts now move meaning instead of jumping slots.

Close Means Similar

Cat near dog scores cosine 0.949 while car scores 0.447.

You dot cat [2,1] with dog [2,2] side by side and get cosine 0.949 near 18 degrees. Cat with car [0,3] gives 0.447 near 63 degrees, so friends sit closer than strangers.

Context Pulls Together

Shared contexts pull cat and dog to nearby coordinates.

You count neighbors and let them tug each dot. Cat and dog share pet, feed, and vet, so training drags them close while car drifts toward road words.

Gaps Tell Truth

Equal gaps become 0.28 for friends and 2.60 for strangers.

You measure the same trio on one ruler in this fixture. One-hot says both gaps equal root 2, while dense says 0.28 for cat-dog and 2.60 for cat-car, which matches your gut.

Pick Your Width

Start near 100 dims and grow only when words still collide.

You start near 100 dims for small vocabs and grow toward 300 for big ones. Flip smaller when training wobbles, since extra slots cost memory without adding sense.

Where It Breaks

One bank vector between finance and river fits neither.

You force bank to one dot for money and river. It settles halfway, 1.3 from each group, and answers both queries badly until you add context.

The Quick Version

One-hot keeps all gaps equal.
100 dense numbers replace 10,000 zeros.
Cosine 0.949 beats 0.447 for closeness.
Shared contexts drag friends together.
0.28 versus 2.60 matches your gut.
Grow dims only when words still collide.
One bank dot fails both meanings.