Glossary
Context window
The maximum number of tokens a model can consider at once.
What it is
The context window is the total tokens (prompt + response) a model can handle in a single call. It bounds how much retrieved context, history, and instructions you can include, which is why retrieval and summarization matter.
Why it matters in production
Bigger windows didn't remove the need for retrieval: stuffing everything in is slow, expensive, and models still lose details in the middle. Treat the window as a budget you spend deliberately — system prompt, retrieved chunks, history, output space.
In practice
A support bot that pastes 40 documents into the prompt pays for 40 documents per question; one that retrieves the 4 best chunks answers faster, cheaper, and usually better.
Related terms
Production AI Notes
One practical AI engineering email each week
One concept, one architecture, one project idea, and one interview question — written for developers who want to build and ship real AI systems.