Autoregressive Model
A type of sequence generation model that predicts the very next token in a series by conditioning its output on all of the previous tokens it generated.
Think of It Like This
Like a person writing a book by only looking at the words they have written so far to decide what the very next single word should be.
Autoregressive architectures like the Transformer decoder process text iteratively, making them the foundational engine behind modern Large Language Models. Because they must generate one token at a time during inference, they are inherently bottlenecked by memory bandwidth rather than pure compute. This sequential generation process contrasts heavily with diffusion models or masked language models that evaluate the entire context simultaneously.