Learn to run ML training pipelines on Kubernetes with Argo Workflows and Kubeflow Pipelines: DAGs, artifacts, retries, gates, caching, schedules.
Tuesday, 1:10 AM. acme-shop's monthly retraining of the fraud model kicks off on the same Airflow worker VMs that run the nightly ETL; acme-shop also...
Argo and Kubeflow use different words, but they are built on the same four ideas. Learn the ideas once and both tools read the same way.
Argo Workflows describes a pipeline as Kubernetes YAML. It is verbose, but nothing is hidden: every pod, volume and retry rule is in the file.
A training pipeline must tell apart "the node died" from "the data is wrong", stop a bad model before it is registered, and run on a schedule.
Kubeflow Pipelines turns Python functions into pipeline steps.
A compiled pipeline runs on the KFP backend in the cluster.
Tuesday, 1:10 AM. acme-shop's monthly retraining of the fraud model kicks off on the same Airflow worker VMs that run...
Argo and Kubeflow use different words, but they are built on the same four ideas.
Argo Workflows describes a pipeline as Kubernetes YAML.
A training pipeline must tell apart "the node died" from "the data is wrong", stop a bad model before it is registered...
Kubeflow Pipelines turns Python functions into pipeline steps.
A compiled pipeline runs on the KFP backend in the cluster.
You will not need GPUs or multi-node training for the labs in this roadmap.
A pipeline that trains a model and leaves it in an artifact store is half done.
A pipeline platform runs arbitrary code from many people with access to training data and the model registry.
This lab runs the same four-step acme-fraud training pipeline (ingest, validate, train, evaluate, then a gated...
Quick reference Common mistakes Adopting Kubeflow to fix a resource problem is the most expensive mistake in this...
Roles that use the skills in this module.
AI Engineer
MLOps Engineer
Platform Engineer
DevOps Engineer
Not a software engineer sheet. Every problem comes from real DevOps, SRE, Platform and Cloud interviews, from your first script to a system you build yourself.
Open the Coding SheetUsually not. Airflow with KubernetesPodOperator already runs each training step in its own pod with its own image and resources, which covers most needs. Kubeflow Pipelines earns its place when many data scientists build pipelines themselves, want artifact lineage and caching built in, and the platform team already runs Kubernetes well.
Argo Workflows is a general Kubernetes workflow engine: you describe steps as containers in YAML and it runs them as pods. Kubeflow Pipelines is an ML platform with a Python SDK, typed artifacts, caching, experiment tracking and a UI. Kubeflow Pipelines has historically used Argo Workflows underneath to execute the steps.
Yes. Both labs run on kind, a Kubernetes cluster inside Docker on a laptop, using CPU only. Give Docker at least 4 CPUs and 8 GB of memory, because Kubeflow Pipelines runs several services of its own.
Kubeflow Pipelines decides whether to reuse a step's output by looking at the step's code, image and inputs. If a step reads data from outside, such as a bucket that changed overnight, its inputs look identical and the old result is reused. Pass a data version as an input or turn caching off for steps that read external data.