ML Systems in Production
Why models degrade after launch, how batch and online inference differ, and what to monitor.
- 4 lessons
- 1h 4m
- 4 simulations
Drift detection, performance with delayed labels, retraining strategies, and safe rollouts.
This is the real simulation from the module. No account needed.
A suggestion, never a lock — you can start here.
Every lesson's title and summary is public. The lessons themselves open once you own the track.
Every lesson in MLOps: Deploying Machine Learning & Deep Learning ModelsPSI with quantile bins, the KS test, why p-values are always "significant" on big data, and how to calibrate thresholds so the alarm does not fire every morning.
When to retrain — on a schedule, on drift, or on performance — on which data window, and why a trigger is not permission to release.
Running a new model beside the old one — with no consequences (shadow), then on a small slice of traffic (canary) — and computing how much traffic it takes before the result can be trusted.
Why models degrade after launch, how batch and online inference differ, and what to monitor.
From notebook to service: saving models, an API with FastAPI, and reproducible Docker containers.
GPUs, batching, quantisation and formats like ONNX — making large models fast enough and cheap enough.
Experiment tracking, model registries, and automated pipelines from raw data to a released model.