How AI context windows work
Tokens, attention and why bigger context windows are not always better.
A model's context window is the number of tokens it can attend to in a single forward pass. Recent frontier models have pushed context to 1M–2M tokens (Gemini 1.5 Pro, Llama 4). But attention cost scales super-linearly and retrieval quality inside long contexts is uneven — the classic "lost in the middle" problem. This guide explains what a token is, how context is used and when RAG still beats simply pasting more into the prompt.