---
title: "Multiple Experiments & Runs"
excerpt: "Learn how to use multiple experiments and runs"
tags: ['mlops', 'mlflow', 'python', 'machine-learning']
---
## 🤖 Creating Multiple Experiments & Runs
You can create many `experiments` and nested/child `runs`. Where:
Experiments
An **experiment** is a container for related machine learning runs. It provides a way to group and compare training attempts for a particular objective.
For example:
- `02_regression_experiment` — experiments with models that predict continuous values.
- `02_classification_experiment` — experiments with models that predict class labels.
Each experiment has a unique experiment ID. Experiments can also have descriptive names to make them easier to identify.
Runs
A **run** represents one execution of a machine learning workflow. A run can record:
- `Parameters`: Configuration values such as n_estimators and max_depth.
- `Metrics`: Evaluation results such as RMSE and accuracy.
- `Tags`: Metadata describing the run, such as its stage, team, or purpose.
- `Artifacts`: Files produced during training, such as model files, plots, and evaluation reports.
For example, training a random forest with 100 trees and a maximum depth of 4 can be recorded as one run. Changing the maximum depth to 6 and training again produces another run.
Nested Runs
A **nested run** is a run created inside another run, known as the parent run. It helps organize related training runs under a common task, such as *hyperparameter tuning*, *model comparison*, or an *experiment with multiple configurations*.
For example, a hyperparameter search may test different values of n_estimators and max_depth. The parent run represents the overall search, while each child run records the results of one parameter combination.
```text
Experiment: 02_regression_experiment
├── Run: rf_baseline
└── Parent Run: rf_hyperparameter_sweep
├── Child Run: rf_50_3
├── Child Run: rf_50_5
├── Child Run: rf_100_3
└── Child Run: rf_100_5
```
## 📝 Example Code
```python showLineNumbers {11-14,22,34,38-42,61-64,72,84,88-92}
import numpy as np
import mlflow
from sklearn.datasets import load_diabetes, load_breast_cancer
from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, accuracy_score
# Initialize MLflow tracking
mlflow.set_tracking_uri("sqlite:///mlflow.db")
# --------------------------------------------------
# 1. Experiment 1: Regression (Baseline & Sweep)
# --------------------------------------------------
mlflow.set_experiment("02_regression_experiment")
X_reg, y_reg = load_diabetes(return_X_y=True, as_frame=True)
X_train_r, X_test_r, y_train_r, y_test_r = train_test_split(
X_reg, y_reg, test_size=0.2, random_state=42
)
# Baseline run
with mlflow.start_run(run_name="rf_regression_baseline"):
mlflow.set_tags({"team": "ml_platform", "stage": "baseline"})
model = RandomForestRegressor(n_estimators=100, max_depth=4, random_state=42)
model.fit(X_train_r, y_train_r)
rmse = float(np.sqrt(mean_squared_error(y_test_r, model.predict(X_test_r))))
mlflow.log_params(
{"model": "RandomForestRegressor", "n_estimators": 100, "max_depth": 4}
)
mlflow.log_metric("rmse", rmse)
# Nested runs: hyperparameter sweep
with mlflow.start_run(run_name="rf_regression_sweep") as parent_run:
mlflow.set_tags({"search_type": "manual_grid", "objective": "minimize_rmse"})
mlflow.log_param("search_space", "n_estimators x max_depth")
for n_estimators in [50, 100]:
for max_depth in [3, 5]:
with mlflow.start_run(
run_name=f"rf_reg_{n_estimators}_{max_depth}", nested=True
):
model = RandomForestRegressor(
n_estimators=n_estimators, max_depth=max_depth, random_state=42
)
model.fit(X_train_r, y_train_r)
rmse = float(
np.sqrt(mean_squared_error(y_test_r, model.predict(X_test_r)))
)
mlflow.log_params(
{
"model": "RandomForestRegressor",
"n_estimators": n_estimators,
"max_depth": max_depth,
}
)
mlflow.log_metric("rmse", rmse)
# --------------------------------------------------
# 2. Experiment 2: Classification (Baseline & Sweep)
# --------------------------------------------------
mlflow.set_experiment("02_classification_experiment")
X_clf, y_clf = load_breast_cancer(return_X_y=True, as_frame=True)
X_train_c, X_test_c, y_train_c, y_test_c = train_test_split(
X_clf, y_clf, test_size=0.2, random_state=42, stratify=y_clf
)
# Baseline run
with mlflow.start_run(run_name="rf_classifier_baseline"):
mlflow.set_tags({"team": "ml_platform", "stage": "baseline"})
classifier = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
classifier.fit(X_train_c, y_train_c)
acc = float(accuracy_score(y_test_c, classifier.predict(X_test_c)))
mlflow.log_params(
{"model": "RandomForestClassifier", "n_estimators": 100, "max_depth": 5}
)
mlflow.log_metric("accuracy", acc)
# Nested runs: hyperparameter sweep for classification
with mlflow.start_run(run_name="rf_classifier_sweep") as parent_run_c:
mlflow.set_tags({"search_type": "manual_grid", "objective": "maximize_accuracy"})
mlflow.log_param("search_space", "n_estimators x max_depth")
for n_estimators in [50, 100]:
for max_depth in [3, 7]:
with mlflow.start_run(
run_name=f"rf_clf_{n_estimators}_{max_depth}", nested=True
):
classifier = RandomForestClassifier(
n_estimators=n_estimators, max_depth=max_depth, random_state=42
)
classifier.fit(X_train_c, y_train_c)
acc = float(accuracy_score(y_test_c, classifier.predict(X_test_c)))
mlflow.log_params(
{
"model": "RandomForestClassifier",
"n_estimators": n_estimators,
"max_depth": max_depth,
}
)
mlflow.log_metric("accuracy", acc)
```
In the code above, we created two experiments: `02_regression_experiment` and `02_classification_experiment`. We also created a nested run for the regression experiment.
```text
02_regression_experiment
├── rf_regression_baseline
├── rf_regression_sweep
├── rf_reg_50_3
├── rf_reg_50_5
├── rf_reg_100_3
└── rf_reg_100_5
```
```text
02_classification_experiment
├── rf_classifier_baseline
├── rf_classifier_sweep
├── rf_clf_50_3
├── rf_clf_50_7
├── rf_clf_100_3
└── rf_clf_100_7
```
## 📊 Visualizing Multiple Experiments & Runs
You can visualize multiple experiments and runs using the MLflow UI.
```bash
mlflow ui
```