AI engineering interviews are less about trivia and more about judgment: can you design, debug, and reason about systems that use models? Here are the questions that come up again and again, with the shape of a strong answer.
RAG & retrieval
"Your RAG app returns confident but wrong answers. How do you debug it?" Start with retrieval, not the model. Log the retrieved context — is the right chunk even fetched? Then check chunking, reranking, and grounding. Only after that, the prompt and model.
"How would you evaluate a RAG system?" Build a small representative test set, do error analysis, add automated checks for the failure modes you find, and track a score over time to catch regressions.
Agents
"When would you use an agent instead of a workflow?" Only when steps can't be predetermined. Workflows are more reliable; reach for a full agent for open-ended, branching, multi-tool tasks — and add guardrails and evals.
System design
"Design a chat-with-your-docs feature for 100k users." Cover ingestion pipeline, chunking, vector store + metadata, retrieval + rerank, grounding with citations, caching, cost/latency budgets, evals, tracing, and prompt-injection defense. Draw the layers; justify trade-offs.
Production & cost
"Your token bill tripled. What do you do?" Trace usage per request, cache repeated calls (semantic caching), trim retrieved context, and tier models (small where possible). Cost is an architecture decision.
How to prepare
Build the projects on the roadmap so your answers come from real experience, then rehearse explaining your architecture and trade-offs out loud. For a structured question bank, project specs, and answer frameworks, the AI Engineer Interview & Portfolio Kit is built for exactly this.