How do I reproduce a machine learning training run?

Summary

• MLflow tracking automatically captures parameters, metrics, artifacts, and dataset lineage needed to recreate any training run exactly.
• Git integration and dataset versioning let you preserve the code and data state at the moment the model was trained.
• Experiments organize runs so you can compare configurations side-by-side and identify which settings produced your best results.
• The MLflow UI displays the full audit trail linking each model to its run, environment, code commit, and input dataset.

How do I reproduce a machine learning training run?

Reproducibility in machine learning requires capturing and preserving the exact configuration, code, and data that produced a model. MLflow tracking makes this automatic by recording parameters, metrics, artifacts, source code information, and dataset lineage for every training run. When you need to recreate a past result, you have the complete record stored in the MLflow tracking backend.

What is a training run?

A training run is a single execution of your model training code. It captures everything that happened during that execution: the input parameters you set, the metrics the model achieved, the artifacts it produced (like the trained model file), and metadata about the code and data used. Each run gets a unique ID, and Databricks stores all this information together so you can revisit and recreate it anytime.

Key components for reproducible runs

Component	Why it matters
Parameters	Hyperparameters and configuration settings define how the model was trained. Recording them means you can retrain with identical settings.
Metrics	Performance measurements (accuracy, loss, etc.) show what your model achieved and help you compare runs.
Artifacts	Output files including the trained model, evaluation plots, and reports are stored with the run for reference and deployment.
Source information	The notebook and Git commit that produced the run creates an audit trail. Versioning your code lets you recover the exact training logic.
Dataset metadata	Logging dataset timestamps and schema ensures you can reconstruct the exact training data, even after downstream changes.
Environment	The Python packages, libraries, and runtime configuration are part of the run record.

How Databricks approaches reproducibility

Automatic logging

Databricks Autologging captures parameters, metrics, and artifacts automatically when you train with popular frameworks like scikit-learn, PyTorch, TensorFlow, and XGBoost. This means you get reproducibility with minimal code changes. The system logs model hyperparameters, training metrics, and the trained model artifact without requiring you to add explicit logging statements.

Explicit logging for full control

For custom training code or when you need to log domain-specific information, use the MLflow logging API to record exactly what matters. Log parameters before training starts, metrics as training progresses, and artifacts when training completes.

Versioning code and data

Store your training notebooks in Git folders (Databricks version control integrated with any Git provider). Each training run automatically records the Git commit hash, so you can check out the exact code that produced a model. For datasets, log dataset timestamps as MLflow parameters. If you use bitemporal tables in Databricks, log both business and system time to capture the exact data state at training time.

Organizing with experiments

Group related runs into experiments in MLflow. This lets you compare runs side-by-side, filter by parameters and metrics, and identify which configuration produced your best results. The Experiment Runs sidebar in Databricks shows all runs in an experiment with their parameters, metrics, and tags.

Retrieving and comparing historical runs

The MLflow UI displays the complete audit trail for every run. You can view parameters, metrics, artifacts, and source information in the Databricks interface. To recreate a past run, examine the logged parameters and retrain with identical settings, or load the logged model artifact directly if you need to evaluate or serve it. The audit trail links each model version to its training run, dataset, environment, and Git commit.

Use cases

Model debugging: When a model's performance unexpectedly degrades in production, reproduce the original training run using the stored parameters and data snapshot to confirm the issue and test fixes.
Compliance and audit: Regulated industries need to prove that a model was trained on specific data with known parameters. MLflow's full audit trail satisfies these requirements.
Team collaboration: Different data scientists can pick up someone else's work by examining that person's run in MLflow and understanding exactly how the model was built.
Experiment iteration: Reproduce promising runs and then make small adjustments to parameters or data to improve results incrementally.

FAQs

Do I need to manually log everything for reproducibility?

No. Databricks Autologging handles it automatically for most frameworks. If you use custom training code, the MLflow logging API lets you add reproducibility without significant effort.

What happens if my source code changes after a run?

The Git commit hash is stored with the run. You can check out that exact commit anytime to get the original code. MLflow doesn't prevent code changes; it just records what was used.

Can I reproduce a run if the underlying data changed?

Yes, if you logged dataset metadata and timestamps. For datasets in Delta Lake, log the data version as an MLflow parameter. After VACUUM operations clean up old files, the logged metadata still exists, allowing you to reconstruct the dataset that was used.

Where can I view and compare my past runs?

The MLflow UI in Databricks shows all runs in an experiment. You can compare parameters, metrics, and artifacts side-by-side to understand which settings produced your best results.

Can I use a past run's artifact directly?

Yes. Every run stores artifacts like the trained model. You can load and use the model artifact from any past run without retraining.