[sdk] Set experiment in model_version run

[leonlnj]

Description

This PR clean up the example notebook and fixes the local pyfunc and remote pyfunc clashes

Running the local pyfunc and then deploying to remote MLP raises exception in Merlin sdk

MlflowException: Cannot start run with ID 48edb0102f24433c8914c85905d9b683 because active run ID does not match environment run ID. Make sure --experiment-name or --experiment-id matches experiment set with set_experiment(), or just use command-line arguments

This happen due to the local pyfunc setting an mlflow experiment

And the current sdk assume there is no active experiments prior to running merlin.new_model_version().start() and mlflow will use the default experiment. However after running the local pyfunc, the experiment is set which leads to this exception from the mlflow client

As _active_experiment_id is not None and _active_experiment_id != active_run_obj.info.experiment_id

Since the mlflow experiment and run is created remotely during new_model_version(), the experiment can be set and match.

Modifications

Add set_experiment to already created experiment and run before set_model in sdk/merlin/model.py
Remove duplicated blocks in the note book and add a section to mlflow.end_run() before running remote

Tests

Tested with the sample notebook

• Deploying local pyfunc and remote pyfunc on existing model
• Deploying local pyfunc and remote pyfunc on new model
• Deploying remote pyfunc on existing model

Checklist

• Added PR label
• Tested locally
• Updated documentation

Release Notes

NONE

[ghost]

👇 Click on the image for a new way to code review

Legend

[tiopramayudi]

LGTM, thanks @leonlnj