Glossary
Chunking
Splitting documents into retrievable pieces before embedding.
What it is
Chunking breaks source documents into passages sized for retrieval and the context window. Chunk size, overlap, and boundaries (semantic vs fixed) strongly affect retrieval quality in a RAG system.
Why it matters in production
Chunking is the highest-leverage RAG knob. Chunks too big dilute relevance and waste tokens; too small lose context. Following document structure (headings, sections) with 10–20% overlap beats fixed character counts almost every time.
In practice
Splitting a handbook by section headings into ~300-token chunks with 15% overlap, keeping source + section metadata on every chunk for citations.
Related terms
Production AI Notes
One practical AI engineering email each week
One concept, one architecture, one project idea, and one interview question — written for developers who want to build and ship real AI systems.