Adaptive Optimizers
Any optimizer that gives each parameter its own effective learning rate from that parameter's own gradient history, rather than one shared rate for all of them.
Adam is the default nearly everyone starts with, keeping two running-average state vectors per parameter. RMSprop keeps one, dropping momentum. Adafactor compresses Adam's per-matrix second moment into row and column factors instead of the full matrix, cutting memory by orders of magnitude on large layers. Lion keeps one state vector and steps by the sign of it rather than its magnitude.
None of these displaces Adam by default — each answers a specific memory or batch-size constraint Adam runs into only at a particular scale.