CODEWITHSUNDEEP
loggingdata-sciencemlflow
mountrix
mountrix

Reputation: 1243

How to update a previous run into MLFlow?

I would like to update previous runs done with MLFlow, ie. changing/updating a parameter value to accommodate a change in the implementation. Typical uses cases:

It is not always easy to trash the whole experiment as I need to keep the previous runs for statistical purpose. I would like also not to generate new experiments just for a single new parameter, to keep a single database of runs.

What is the best way to do this?

Upvotes: 20

Views: 15412

Answers (3)

Moein Hasani
Moein Hasani

Reputation: 91

Well you can't update it but there is another solution which is getting the run, edit the value that you want, and then log the run as a whole new run.

import mlflow 
import os 
os.environ["MLFLOW_EXPERIMENT_NAME"] = "REPLACE_YOUR_EXPERIEMNTE_NAME"

original_run_id = 'REPLACE_YOUR_RUN_ID'
original_run = mlflow.get_run(original_run_id)
original_run_name = original_run.data.tags.get('mlflow.runName')

with mlflow.start_run(run_name=original_run_name) as new_run:
    for key, value in original_run.data.params.items():
        if key == 'REPLACE_WITH_YOUR_KEY':
            new_value = REPLACE_WITH_YOUR_VALUE
            mlflow.log_param(key, new_value)
        else:
            mlflow.log_param(key, value)

    for key, value in original_run.data.metrics.items():
        mlflow.log_metric(key, value)

setting the epxeriemnt name and run name are optional, the run_id should grab the exact run that you want. But setting the run name will help you create a name the same as before. This snippet currently updates a param, you can change it to update a metric.

Upvotes: 0

Pan
Pan

Reputation: 938

MLflow API does not support updating an existing parameter value, see this.

However, there are backdoors you can use to achieve the goal of rewriting an existing parameter's value. But use with caution.

Upvotes: 1

user20061
user20061

Reputation: 554

To add or correct a parameter, metric or artifact of an existing run, pass run_id instead of experiment_id to mlflow.start_run function

with mlflow.start_run(run_id="your_run_id") as run:
    mlflow.log_param("p1","your_corrected_value")
    mlflow.log_metric("m1",42.0) # your corrected metrics
    mlflow.log_artifact("data_sample.html") # your corrected artifact file

You can correct, add to, or delete any MLflow run any time after it is complete. Get the run_id either from the UI or by using mlflow.search_runs.

Source: https://towardsdatascience.com/5-tips-for-mlflow-experiment-tracking-c70ae117b03f

Upvotes: 24

Related Questions