Merge branch 'main' into secret_manager

build.sbt

@@ -2,7 +2,7 @@ import sbt.Keys.publishLocalConfiguration
 
 ThisBuild / resolvers += Resolver.mavenLocal
 ThisBuild / scalaVersion     := "2.12.15"
-ThisBuild / version          := "0.8.2"
+ThisBuild / version          := "0.8.0"
 ThisBuild / organization     := "com.linkedin.feathr"
 ThisBuild / organizationName := "linkedin"
 val sparkVersion = "3.1.3"

docs/concepts/feature-registry.md

@@ -74,6 +74,8 @@ client.register_features()
 all_features = client.list_registered_features(project_name=client.project_name)
 ```
 
+Please avoid applying a same name to different features under a certain project. Since it will be treated as updating an exsiting project which is not supported by feathr and will cause errors.
+
 ### Reuse Features from Existing Registry
 
 The feature producers can just let the feature consumers know which features exist so the feature consumers can reuse them. For feature consumers, they can reuse existing features from the registry. The whole project can be retrieved to local environment by calling this API `client.get_features_from_registry` with a project name. This encourage feature reuse across organizations. For example, end users of a feature just need to read all feature definitions from the existing projects, then use a few features from the projects and join those features with a new dataset you have.

docs/dev_guide/feathr_overall_release_guide.md

@@ -31,11 +31,11 @@ Read through the [commit log](https://github.com/feathr-ai/feathr/commits/main)
 ## Code Changes
 Before the release is made, the version needs to be updated in following places
 - [build.sbt](https://github.com/feathr-ai/feathr/blob/main/build.sbt#L3) - For Maven release version
-- [setup.py](https://github.com/feathr-ai/feathr/blob/main/feathr_project/setup.py#L10) - For PyPi release version
+- [version.py](https://github.com/feathr-ai/feathr/blob/main/feathr_project/feathr/version.py#L1) - For Feathr version
 - [conf.py](https://github.com/feathr-ai/feathr/blob/main/feathr_project/docs/conf.py#L27) - For documentation version
-- [feathr_config.yaml](https://github.com/feathr-ai/feathr/blob/main/feathr_project/test/test_user_workspace/feathr_config.yaml#L84) - To set the spark runtime location for Azure Synapse and Azure Databricks used by test suite.
-- [constants.py](https://github.com/feathr-ai/feathr/blob/73656fe4a57219e99ff6fede10d51a000ae90fa1/feathr_project/feathr/constants.py#L31) - To set the default maven artifact version
+- [feathr_config.yaml](https://github.com/feathr-ai/feathr/blob/main/feathr_project/test/test_user_workspace/feathr_config.yaml#L84) - To set the spark runtime location for Azure Synapse and Azure Databricks used by test suite. Please update all .yaml files under this path. 
 - [azure_resource_provision.json](https://github.com/feathr-ai/feathr/blob/main/docs/how-to-guides/azure_resource_provision.json#L114) - To set the deployment template to pull the latest release image.
+- [constants.py](https://github.com/feathr-ai/feathr/blob/main/feathr_project/feathr/constants.py#L31) - To set the default maven artifact version (Only needed when maven version is **NOT** the same as python sdk version)
 
 ## Triggering automated release pipelines
 Our goal is to automate the release process as much as possible. So far, we have automated the following steps

docs/how-to-guides/azure_resource_provision.json

@@ -111,7 +111,7 @@
       "destinationBacpacBlobUrl": "[concat('https://',variables('dlsName'),'.blob.core.windows.net/',variables('dlsFsName'),'/',variables('bacpacBlobName'))]",
       "bacpacDeploymentScriptName": "CopyBacpacFile",
       "bacpacDbExtensionName": "registryRbacDbImport",
-      "preBuiltdockerImage": "feathrfeaturestore/feathr-registry:releases-v0.8.2"
+      "preBuiltdockerImage": "feathrfeaturestore/feathr-registry:releases-v0.8.0"
     },
     "functions": [],
     "resources": [

docs/how-to-guides/feathr-configuration-and-env.md

@@ -116,7 +116,7 @@ feathr_client = FeathrClient(..., secret_manager_client = cache)
 | SPARK_CONFIG__SPARK_RESULT_OUTPUT_PARTS             | Configure number of parts for the spark output for feature generation job                                                                                                                                                                                                          | Required                                                                  |
 | SPARK_CONFIG__AZURE_SYNAPSE__DEV_URL                  | Dev URL to the synapse cluster. Usually it's something like `https://yourclustername.dev.azuresynapse.net`                                                                                                                                                                         | Required if using Azure Synapse                                           |
 | SPARK_CONFIG__AZURE_SYNAPSE__POOL_NAME                | name of the spark pool that you are going to use                                                                                                                                                                                                                                   | Required if using Azure Synapse                                           |
-| SPARK_CONFIG__AZURE_SYNAPSE__WORKSPACE_DIR            | A location that Synapse has access to. This workspace dir stores all the required configuration files and the jar resources. All the feature definitions will be uploaded here                                                                                                     | Required if using Azure Synapse                                           |
+| SPARK_CONFIG__AZURE_SYNAPSE__WORKSPACE_DIR            | A location that Synapse has access to. This workspace dir stores all the required configuration files and the jar resources. All the feature definitions will be uploaded here. Suggest to use an empty dir for a new spark job to avoid conflicts.                                                                                                     | Required if using Azure Synapse                                           |
 | SPARK_CONFIG__AZURE_SYNAPSE__EXECUTOR_SIZE            | Specifies the executor size for the Azure Synapse cluster. Currently the options are `Small`, `Medium`, `Large`.                                                                                                                                                                   | Required if using Azure Synapse                                           |
 | SPARK_CONFIG__AZURE_SYNAPSE__EXECUTOR_NUM             | Specifies the number of executors for the Azure Synapse cluster                                                                                                                                                                                                                    | Required if using Azure Synapse                                           |
 | SPARK_CONFIG__AZURE_SYNAPSE__FEATHR_RUNTIME_LOCATION  | Specifies the Feathr runtime location. Support local paths, path start with `http(s)://`, and paths start with `abfss:/`. If not set, will use the [Feathr package published in Maven](https://search.maven.org/artifact/com.linkedin.feathr/feathr_2.12).                         | Required if using Azure Synapse                                           |