From 98b0dd6bc7bac0bad76fade209dd580553288239 Mon Sep 17 00:00:00 2001 From: Eric Li Dong <97083728+eric-lidong@users.noreply.github.com> Date: Mon, 19 Feb 2024 11:31:29 +0800 Subject: [PATCH] Add MLObs onboarding docs to gitbook (#538) # Description Adding gitbook reference to the MLobs related documentation, including the model schema and the mlobs onboarding guide. # Modifications Add gitbook summary reference for new docs, fixed one heading typo in mlobs onboarding guide. # Tests --> # Checklist - [ ] Added PR label - [ ] Added unit test, integration, and/or e2e tests - [ ] Tested locally - [ ] Updated documentation - [ ] Update Swagger spec if the PR introduce API changes - [ ] Regenerated Golang and Python client if the PR introduces API changes # Release Notes ```release-note ``` --- docs/SUMMARY.md | 3 +++ docs/user/generated/11_model_observability.md | 5 ++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 7faf2b789..d34627b6c 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -21,6 +21,8 @@ * [Deleting a Model](/user/generated/04_deleting_a_model.md) * [Configuring Alerts](/user/generated/05_configuring_alerts.md) * [Batch Prediction](/user/generated/06_batch_prediction.md) +* [Model Schema](/user/generated/10_model_schema.md) +* [Model Observability](/user/generated/11_model_observability.md) * [Model Sample Notebooks](/user/generated/07_examples.md) * [Deploy Standard Models](/user/generated/examples/01_standard_model.md) * [Deploy PyFunc Model](/user/generated/examples/02_pyfunc_model.md) @@ -28,6 +30,7 @@ * [Run Batch Prediction Job](/user/generated/examples/04_batch_prediction.md) * [Others examples on Models](/user/generated/examples/05_others.md) * [Limitations](/user/generated/08_limitations.md) +* [Troubleshooting Deployment Errors](/user/generated/09_troubleshooting_deployment_errors.md) ## Developer Guides diff --git a/docs/user/generated/11_model_observability.md b/docs/user/generated/11_model_observability.md index 9ca9152c1..533294110 100644 --- a/docs/user/generated/11_model_observability.md +++ b/docs/user/generated/11_model_observability.md @@ -19,7 +19,7 @@ From above architecture diagram, we can see that there are three places where th ## Onboarding As the architecture diagram illustrate, the end to end model onboarding to model observability needs to involving several components. The scope of this section is limited to merlin model modification. -### ### PyFunc modification +### PyFunc modification Currently the only supported model for model observability is PyFunc model, the model should implements class `PyFuncV3Model` instead of `PyFuncModel`. This `PyFuncV3Model` has difference method signature that must be implemented. Following are the new methods: | Method Name | Description | |-------------|-------------| @@ -45,8 +45,7 @@ Beside changes in signature, you can see some of those methods returning new typ Same like `ModelInput`, `ModelOutput` is also essential for model observability, it can be used to calculate prediction drift but more importantly it can calculate performance metrics. ### Configure Model Schema - -Model schema is essential for model observability because it is used by the kafka consumer to choose which columns that is relevant to model observability and do necessary preprocessing before publishing the data to model observability system. Users can see more detail of configuring model schema [here](../templates/09_model_schema.md) +Model schema is essential for model observability because it is used by the kafka consumer to choose which columns that is relevant to model observability and do necessary preprocessing before publishing the data to model observability system. Users can see more detail of configuring model schema [here](../generated/10_model_schema.md) ### Deployment There is not much change on the deployment part, users just needs to set `enable_model_observability` parameter to `True` during model deploy. For clarity, we take one use case for model observability example, suppose a model has 4 features: