Causal Language Model
A neural architecture that predicts the next token in a sequence by strictly masking out all future tokens to prevent any forward-looking data leakage.
Think of It Like This
Like trying to guess the next word in a sentence while holding a piece of paper that completely covers up the rest of the page.
Unlike BERT, which has bidirectional visibility over the entire text, models like GPT can only 'see' the tokens that occurred before the current position. This causal masking constraint forces the model to actually learn the underlying probability distribution of language rather than just filling in blanks, making it exceptionally powerful for open-ended text generation.