Glossary
Token
The unit of text an LLM reads and generates (roughly ¾ of a word).
What it is
A token is the chunk of text a model processes — often a word piece. Pricing, context limits, and latency are all measured in tokens, so token budgeting is a core production concern.
Why it matters in production
You pay per token, you wait per token, and your context window is a token budget. Cost estimates, truncation bugs, and 'why did the model forget the beginning of the conversation' all trace back to token counts.
In practice
A 1,000-word English document is roughly 1,300–1,500 tokens — so a 100-page knowledge base does not fit in the prompt, which is why RAG exists.
Related terms
Production AI Notes
One practical AI engineering email each week
One concept, one architecture, one project idea, and one interview question — written for developers who want to build and ship real AI systems.