Searching, Comparing & Nested Runs

Hyperparameter sweeps create hundreds of runs — find the winner programmatically.

0/2 done

Query, don't scroll

Runs are queryable data

Every run is a row you can query with mlflow.search_runs, which returns a pandas DataFrame. This turns 'which config won?' from UI scrolling into one expression:

import mlflow
df = mlflow.search_runs(
    experiment_names=['credit-scoring'],
    filter_string="metrics.roc_auc > 0.9 and params.model = 'xgboost'",
    order_by=['metrics.roc_auc DESC'],
    max_results=10,
)
best_run_id = df.iloc[0]['run_id']

Nested runs keep sweeps tidy

A hyperparameter sweep should be one parent run with a child run per trial, so the experiment list isn't flooded:

with mlflow.start_run(run_name='sweep') as parent:
    for lr in [0.01, 0.1, 0.3]:
        with mlflow.start_run(nested=True, run_name=f'lr={lr}'):
            mlflow.log_param('lr', lr)
            mlflow.log_metric('roc_auc', train(lr))

The parent groups the trials; search_runs can still reach every child for the comparison.

Analogy

Treat runs like rows in a database, not sticky notes on a monitor. search_runs is your SELECT ... WHERE ... ORDER BY; nested runs are the folder that keeps one sweep's 200 trials from burying every other experiment.

Reflect

Think about your last messy sweep.

  • Could a single `search_runs` filter have found the winner you hunted for by hand?
  • Which two metrics should you order by to avoid shipping an accurate-but-slow model?

Reading in progress · 0 of 2 activities done