Learn to version data and models with DVC: pointer files, remote storage, reproducible pipelines, metric comparison, and tracing a model to its data.
It is Thursday at acme-shop. A teammate says the fraud model got worse after last week's retrain. You ask which data it was trained on.
You will now put the payments file under version control. The commands are short. What matters is understanding what each one creates.
The cache lives on your own machine. If your laptop breaks, or a teammate needs the data, the cache does not help. You need a shared place.
Tracking data solves half of the problem.
Once settings, data and results are all versioned, you can compare any two points in your history.
DVC is most useful when more than one person touches the data. Branches and data versions A data change is a Git change.
It is Thursday at acme-shop. A teammate says the fraud model got worse after last week's retrain.
You will now put the payments file under version control. The commands are short.
The cache lives on your own machine. If your laptop breaks, or a teammate needs the data, the cache does not help.
Tracking data solves half of the problem.
Once settings, data and results are all versioned, you can compare any two points in your history.
DVC is most useful when more than one person touches the data. Branches and data versions A data change is a Git change.
Versioned data is also data you must look after, and data that can answer an important question.
DVC is a good first choice, but it is not the only one.
You will version the payments data, share it through a remote, build the pipeline, change the data and the settings...
Quick reference Common mistakes Editing a tracked data file and committing without dvc add leaves the pointer...
Roles that use the skills in this module.
AI Engineer
MLOps Engineer
Platform Engineer
Data Engineer
Not a software engineer sheet. Every problem comes from real DevOps, SRE, Platform and Cloud interviews, from your first script to a system you build yourself.
Open the Coding SheetGit is built for source code and small text files, and it keeps every version inside the repository. Large datasets and binary files that change often make a repository much bigger, slower to clone and harder to maintain, and Git cannot show a useful difference for them. DVC keeps a tiny pointer file in Git and stores the real data in a cache and a remote, so Git stays small and fast.
No. DVC keeps large data and model files outside Git. Git stores the DVC pointer files and pipeline files, next to your normal code, settings and any small metrics files you choose to commit. The data itself goes to a local cache and to a remote storage that you choose, such as a shared folder or an S3 bucket. Git history then tells you which version of the data belongs to which commit.
DVC remembers what each stage used: the data, the code and the parameters. When you run the pipeline again, it only runs the stages whose inputs changed, and it takes the others from its cache. This saves time and records which tracked inputs and parameters produced each output, so you have a written record instead of relying on memory or file names. It does not prove that training is deterministic, or that the data is correct.
Both can, but they do different jobs. DVC versions the data and the pipeline, and can run quick experiments. MLflow, in the next module, keeps a searchable history of runs and a registry of models. Many teams use both: DVC answers which data was used, and MLflow answers which run was best.