Paolo Di Francesco

Paolo Di Francesco

MLflow contributor · MLOps & agentic AI at AWS

Track your ML experiments end to end with Data Version Control and Amazon SageMaker Experiments

less than 1 minute read

Data scientists often work towards understanding the effects of various data preprocessing and feature engineering strategies in combination with different model architectures and hyperparameters. Doing so requires you to cover large parameter spaces iteratively, and it can be overwhelming to keep track of previously run configurations and results while keeping experiments reproducible.

Full text here, and GitHub repository here

Architecture: a SageMaker Studio notebook configures Git and DVC in AWS CodeCommit, launches a SageMaker Processing job that stores the versioned dataset in Amazon S3 and syncs its metadata with Git, then launches a SageMaker Training job that pulls the data from S3 and records the Git commit in SageMaker Experiments

This post walks through how to use Data Version Control (DVC) alongside Amazon SageMaker Experiments to track data and code changes together with ML experiment metrics, providing end-to-end traceability for your ML workflows.

2026 update. SageMaker Experiments has since been superseded by managed MLflow on Amazon SageMaker AI as the experiment-tracking layer. The core idea — versioning data and code together with experiment metrics — still applies unchanged; for the current take on it, see the successor post: End-to-end lineage with DVC and SageMaker AI MLflow apps.

You May Also Enjoy

Govern models with MLflow and Amazon SageMaker AI Model Registry sync: Part 2

Larger organizations rarely stop at one account: they run multiple development accounts, a central governance function, and — in regulated environments — a hard rule that development workloads cannot write into production-grade accounts. In this second post, we extend the building blocks from Par...

Govern models with MLflow and Amazon SageMaker AI Model Registry sync: Part 1

Data scientists track dozens of candidate runs in MLflow, while governance officers need one authoritative registry to validate, approve, and audit the models that reach production. Managed MLflow on Amazon SageMaker AI now closes that gap: every model registered in MLflow syncs into the SageMake...

Deploying MLflow models to Amazon SageMaker AI Hosting

Your best model sits in the MLflow Model Registry with a version number and a signature — and then someone asks: “Can we get that behind an endpoint?” With managed MLflow on Amazon SageMaker AI, the distance between “registered in MLflow” and “serving traffic” is much shorter than it used to be. ...