A free sample of the full AI Engineer Interview Kit: 8 real questions with model answers, plus the system-design template used in every mock interview.
Sample questions
1. "Walk me through how you'd design a RAG system for customer support tickets."
What they're testing: whether you reach for architecture before code.
Strong answer shape: ingestion (source, chunking strategy, refresh cadence) → retrieval (hybrid search, filters for account/permission) → reranking → generation (grounded prompt, citations) → evals (recall@k, answer accuracy) → observability (tracing, cost, latency). Name the failure modes you'd design around: stale data, no-context queries, prompt injection from ticket content.
2. "Your RAG system's answers got worse after last week's deploy. How do you debug it?"
What they're testing: whether you can isolate retrieval failures from generation failures instead of guessing.
Strong answer shape: check the eval suite first (did a metric regress?), then split the failure: is retrieval returning the right chunks (recall) or is generation ignoring good context (faithfulness)? Check what changed in the deploy — chunking, prompt, model version, embedding model. Reproduce with a labeled failing query before touching anything.
3. "How do you decide between fine-tuning and RAG for a given problem?"
What they're testing: judgment, not a memorized rule.
Strong answer shape: RAG for knowledge that changes often or must cite sources; fine-tuning for consistent style/format/behavior that's hard to prompt into existence. Most production systems need RAG first — it's cheaper, more auditable, and easier to update. Mention cost and latency trade-offs.
4. "Design a function-calling agent that can look up an order and issue a refund."
What they're testing: whether you design guardrails, not just happy path.
Strong answer shape: define tools with strict schemas, validate arguments before execution, add a confirmation step (or a hard cap) before any money-moving action, log every tool call with inputs/outputs, and design for partial failure (the lookup succeeds, the refund call times out — what happens?).
5. "How do you evaluate an LLM system that doesn't have one right answer?"
What they're testing: whether you know eval methods beyond "looks good to me."
Strong answer shape: rubric-based scoring for open-ended output, an LLM-as-judge with a calibrated rubric (validated against human labels), pairwise comparison for regressions, and hard metrics wherever possible (retrieval recall, citation accuracy, latency, cost).
6. "What's your approach to prompt injection?"
What they're testing: whether you treat untrusted content (retrieved docs, tool outputs, user input) as attacker-controlled.
Strong answer shape: never let retrieved/tool content carry instructions the model will follow blindly — separate instructions from data in the prompt, constrain what tools the model can call in response to document content, validate/allowlist outputs before they trigger actions, and log/alert on suspicious patterns.
7. "Tell me about a project where the AI system failed in production. What did you do?"
What they're testing: whether you've actually operated something, not just built a demo.
Strong answer shape: a specific incident, the detection signal (an alert, a user report, an eval regression), the root cause, the fix, and — the part people skip — what you changed so it can't happen the same way again.
8. "How would you control cost for an LLM feature at scale?"
What they're testing: whether you think about unit economics.
Strong answer shape: cache aggressively (exact and semantic), pick the smallest model that meets the quality bar, batch where latency allows, set per-user/per-feature budgets with alerts, and measure cost per successful outcome, not just per call.
System-design answer template
Use this shape for any "design a system that uses an LLM to do X" question:
- Clarify the goal and constraints — scale, latency budget, data sensitivity, accuracy bar. Ask before designing.
- Draw the data flow — input → retrieval/tools → model → output. Say it out loud even without a whiteboard.
- Name the failure modes — what breaks this, and how would you know?
- Evals — how do you measure if this system is actually working?
- Cost and latency — rough numbers, not just "it depends."
- Trade-offs you'd revisit — show you know this design isn't the only one.
The full kit has 40+ questions across RAG, agents, evals, and system design, plus rehearsed STAR stories and a portfolio review checklist. See the AI Engineer Interview & Portfolio Kit.