Glossary
LLMOps
Operating LLM systems: evals, tracing, cost, and reliability.
What it is
LLMOps applies operational discipline to LLM applications: versioning prompts, tracing requests, running evals in CI, monitoring cost and latency, and managing rollouts and incidents.
Why it matters in production
This is where DevOps instincts transfer directly: observability, CI gates, budgets, rollback. Teams hiring for 'AI engineer' increasingly mean 'someone who can operate these systems' — LLMOps is the skill that separates builders from operators.
In practice
Tracing every LLM call with prompt version, tokens, latency, and cost; alerting when p95 latency or daily spend crosses a threshold.
Go deeper
Related terms
Production AI Notes
One practical AI engineering email each week
One concept, one architecture, one project idea, and one interview question — written for developers who want to build and ship real AI systems.