Context Window
The strictly enforced theoretical limit on the total exact number of consecutive tokens a model can simultaneously process and remember during inference.
Think of It Like This
Like a person's short term working memory; you can only hold a certain amount of facts in your head at once before you start forgetting the first things you read.
Because standard attention mechanisms scale quadratically in memory and compute, most transformers max out around a few thousand tokens. If a user pastes an entire book that exceeds this strict window, the model simply cannot see the beginning of the text anymore. Recent innovations like Ring Attention attempt to stretch this window to millions of tokens.