Hashing Trick
Mapping categories or tokens into a fixed number of buckets with a hash function, so the feature width never grows.
Pick a width , hash each value, and use the result modulo as the column index. You decide the number of columns before seeing any data, which is what makes it work on streaming input and on categories that keep appearing.
Two things it buys beyond width. There's no vocabulary to fit, so nothing has to be learned or stored, and an unseen category needs no special handling — it just lands in whichever bucket its hash picks.
The cost is collisions. Two unrelated values sharing a bucket become indistinguishable to the model, and you can't map a coefficient back to a value, so interpretability goes entirely. Raising trades memory for fewer collisions, and the usual rule is to pick several times larger than the number of values you expect to matter.