GlossaryAgents
Context window
The maximum amount of text (in tokens) an LLM can attend to at once.
The context window is the LLM's working memory. Modern models range from 8k tokens (older OSS models) to 1M+ (Claude, Gemini). Bigger isn't always better: long contexts are slower, more expensive, and quality degrades for content in the middle of the window (the "lost in the middle" problem). Good RAG aims to put only the most relevant retrieved content in the window.