Skip to content

Commit

Permalink
[sdk] Set experiment in model_version run (#532)
Browse files Browse the repository at this point in the history
# Description
This PR clean up the example notebook and fixes the local pyfunc and
remote pyfunc clashes

Running the local pyfunc and then deploying to remote MLP raises
exception in Merlin sdk
```
MlflowException: Cannot start run with ID 48edb0102f24433c8914c85905d9b683 because active run ID does not match environment run ID. Make sure --experiment-name or --experiment-id matches experiment set with set_experiment(), or just use command-line arguments
```
This happen due to the local pyfunc [setting an mlflow experiment
](https://github.com/caraml-dev/merlin/blob/2951a2276512ddd9b804cdb4381c03980e1e1b5d/python/sdk/merlin/pyfunc.py#L429)

And the current sdk assume there is no active experiments prior to
running `merlin.new_model_version().start()` and mlflow will use the
`default experiment`. However after running the local pyfunc, the
experiment is set which leads to this exception from the [mlflow
client](https://github.com/mlflow/mlflow/blob/d42864b0168ef328ae9aec6bbe39e05a0c0f76fe/mlflow/tracking/fluent.py#L255)

As `_active_experiment_id is not None and _active_experiment_id !=
active_run_obj.info.experiment_id`

Since the mlflow experiment and run is created remotely during
`new_model_version()`, the experiment can be set and match.

![image](https://github.com/caraml-dev/merlin/assets/30390872/144d0c33-bfa4-427e-9d0e-bb4b5ffdf30a)


# Modifications
Add `set_experiment` to already created experiment and run before
`set_model` in `sdk/merlin/model.py`
Remove duplicated blocks in the note book and add a section to
`mlflow.end_run()` before running remote

# Tests
Tested with the sample notebook
- [x] Deploying local pyfunc and remote pyfunc on existing model
- [x] Deploying local pyfunc and remote pyfunc on new model
- [x] Deploying remote pyfunc on existing model


# Checklist
- [x] Added PR label
- [x] Tested locally
- [x] Updated documentation

# Release Notes

```release-note
NONE
```
  • Loading branch information
leonlnj authored Feb 8, 2024
1 parent 2951a22 commit e110a96
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 81 deletions.
Loading

0 comments on commit e110a96

Please sign in to comment.