The Production RAG Checklist

The checklist for taking RAG from demo to production — chunking, retrieval, reranking, evals, and cost controls.

Most RAG demos work on the happy path and fall apart in production. This is the checklist to run before you call a retrieval system "done."

1 · Chunking and ingestion

Chunks follow document structure (headings, sections) — not fixed character counts.
Chunk size matches the query type (short chunks for facts, larger for narrative/context).
Overlap is set (10–20%) so answers near a chunk boundary aren't lost.
Metadata is preserved per chunk (source, section, date, permissions).
Re-ingestion is idempotent — re-running the pipeline doesn't duplicate vectors.

2 · Retrieval

Hybrid search (vector + keyword/BM25) is available, not vector-only.
Top-k is tuned per use case, not left at a framework default.
Retrieval is tested against a labeled query set, not eyeballed.
Filters (metadata, permissions, recency) are applied at query time.

3 · Reranking and grounding

A reranking step sits between retrieval and generation for high-stakes queries.
The prompt instructs the model to answer only from retrieved context.
The model returns citations, and the UI can show the source chunk.
There's an explicit "I don't know" path when retrieval returns nothing relevant.

4 · Evals

A labeled eval set exists (real or representative queries + expected answers).
Retrieval quality is measured separately from generation quality (recall@k vs. answer correctness).
Evals run automatically on every change to prompts, chunking, or the retrieval config.
Regressions block deploys, not just get logged.

5 · Cost and latency

You know the cost per query (embedding + retrieval + generation).
Caching is in place for repeated/similar queries.
p50/p95 latency is tracked, not just average.
There's a budget alert before costs surprise you in production.

6 · Safety and observability

Inputs are validated before hitting the retrieval/generation pipeline.
Prompt-injection from retrieved documents is considered (a malicious doc shouldn't hijack the response).
Every query is traced: input, retrieved chunks, prompt, output, latency, tokens.
You can reconstruct why the system gave a specific answer after the fact.

What "production-ready" looks like

If you can check every box above, your RAG system is no longer a demo — it's a system you can trust with real users and defend in an interview. If you can't, that's your next sprint, in priority order: chunking → retrieval → evals → everything else.

Want the deeper version — architecture diagrams, code patterns, and the eval harness itself? That's in the AI Engineer Interview & Portfolio Kit.