Context window

How it works

The context window is the maximum number of tokens a model can work with in a single request. On most APIs it includes everything you send (system prompt, instructions, documents, tool definitions, earlier turns) plus the tokens the model generates in its reply, including reasoning tokens.

Window sizes vary widely between models and sometimes between tiers of the same model. Some makers charge a higher rate once a prompt passes a set length, and recall of details placed deep inside a very long prompt can be weaker than for short ones. APIs usually return an error when you exceed the window, while chat apps may silently drop or summarize older turns.

A large window lets you pass whole documents or codebases without building retrieval, but every token is billed on every call. For large contexts that repeat, pair the window with prompt caching, or use retrieval to send only the relevant parts.

Worked example

With an illustrative 200K-token window, a 150K-token codebase plus a 10K-token system prompt leaves 40K tokens. Asking for up to 64K output tokens in that request can fail or get the reply cut short, because 150K + 10K + 64K = 224K, more than 200K.

Common mistakes

• 1Filling the window with input and leaving no room for the reply and its reasoning tokens.
• 2Assuming the model uses every part of a very long prompt equally well; test recall on details placed deep in the context.
• 3Re-sending a huge context on every turn of an agent loop without prompt caching, which multiplies input cost.