MLOps case studies ⚙️
Applied MLOps interviews are rarely definition questions. You get a system, a symptom, and the question "what do you check first?" — and the ordering of your checks is the answer. Each study below gives you the setup, the symptom, a diagnostic order, the actual root cause, and the follow-ups an interviewer will push on.
The model is 0.91 offline and 0.62 in production
Offline AUC never moved. Online CTR fell off a cliff on day one. Nothing about the model changed.
Work through it →
Fraud detection · delayed ground truthThe fraud model looks perfect and is not
Precision monitoring showed 0.97 for three weeks. Chargebacks were up 40% the whole time.
Work through it →
Cost · pipeline orchestrationThe retraining bill tripled and nobody changed the model
Same model, same data volume, same instance type. The monthly GPU spend went from $9k to $31k.
Work through it →
LLM in production · retrievalThe RAG assistant started confidently making things up
No prompt change, no model change. Answer quality collapsed the week after a routine re-index.
Work through it →
Deployment strategy · evaluationThe canary was green and the rollout still failed
5% canary for 24 hours, all metrics flat. At 100% the model degraded within an hour.
Work through it →
Governance · lineageNobody can reproduce the model that is in production
The regulator asked how a decision was made. The answer took eleven days and was 'approximately'.
Work through it →