Skip to content

Commit

Permalink
update environment setup docs
Browse files Browse the repository at this point in the history
Signed-off-by: Matt Green <[email protected]>
  • Loading branch information
emgeee committed Nov 28, 2024
1 parent d06fb2a commit b3b738f
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 40 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -440,11 +440,10 @@ test-python-universal-spark:

### 7. Dependencies

Add any dependencies for your offline store to our `sdk/python/setup.py` under a new `<OFFLINE_STORE>__REQUIRED` list with the packages and add it to the setup script so that if your offline store is needed, users can install the necessary python packages. These packages should be defined as extras so that they are not installed by users by default. You will need to regenerate our requirements files. To do this, create separate pyenv environments for python 3.8, 3.9, and 3.10. In each environment, run the following commands:
Add any dependencies for your offline store to our `sdk/python/setup.py` under a new `<OFFLINE_STORE>__REQUIRED` list with the packages and add it to the setup script so that if your offline store is needed, users can install the necessary python packages. These packages should be defined as extras so that they are not installed by users by default. You will need to regenerate our requirements files:

```
export PYTHON=<version>
make lock-python-ci-dependencies
make lock-python-ci-dependencies-all
```

### 8. Add Documentation
Expand Down
61 changes: 24 additions & 37 deletions docs/project/development-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ See [Contribution process](./contributing.md) and [Community](../community.md) f
## Making a pull request
We use the convention that the assignee of a PR is the person with the next action.

If the assignee is empty it means that no reviewer has been found yet.
If a reviewer has been found, they should also be the assigned the PR.
If the assignee is empty it means that no reviewer has been found yet.
If a reviewer has been found, they should also be the assigned the PR.
Finally, if there are comments to be addressed, the PR author should be the one assigned the PR.

PRs that are submitted by the general public need to be identified as `ok-to-test`. Once enabled, [Prow](https://github.com/kubernetes/test-infra/tree/master/prow) will run a range of tests to verify the submission, after which community members will help to review the pull request.
Expand Down Expand Up @@ -120,51 +120,39 @@ Note that this means if you are midway through working through a PR and rebase,

## Feast Python SDK and CLI
### Environment Setup
Setting up your development environment for Feast Python SDK and CLI:
1. Ensure that you have Docker installed in your environment. Docker is used to provision service dependencies during testing, and build images for feature servers and other components.
#### Tools
- Docker: Docker is used to provision service dependencies during testing, and build images for feature servers and other components.
- Please note that we use [Docker with BuiltKit](https://docs.docker.com/develop/develop-images/build_enhancements/).
- _Alternatively_ - To use [podman](https://podman.io/) on a Fedora or RHEL machine, follow this [guide](https://github.com/feast-dev/feast/issues/4190)
2. Ensure that you have `make` and Python (3.9 or above) installed.
3. _Recommended:_ Create a virtual environment to isolate development dependencies to be installed
```sh
# create & activate a virtual environment
python -m venv venv/
source venv/bin/activate
```
4. (M1 Mac only): Follow the [dev guide](https://github.com/feast-dev/feast/issues/2105)
5. Install uv. It is recommended to use uv for managing python dependencies.
- `make` is used to run various scripts
- [uv](https://docs.astral.sh/) for managing python dependencies. [installation instructions](https://docs.astral.sh/uv/getting-started/installation/)
- (M1 Mac only): Follow the [dev guide if you have issues](https://github.com/feast-dev/feast/issues/2105)
- (Optional): Node & Yarn (needed for building the feast UI)
- (Optional): [Pixi](https://pixi.sh/latest/) for recompile python lock files. Only when you make changes to requirements or simply want to update python lock files to reflect latest versioons.

### Quick start
- create a new virtual env: `uv venv --python 3.11` (Replace the python version with your desired version)
- activate the venv: `source venv/bin/activate`
- Install dependencies `make install-python-dependencies-dev`

### building the UI
```sh
curl -LsSf https://astral.sh/uv/install.sh | sh
```
or
```ssh
pip install uv
```
6. (Optional): Install Node & Yarn. Then run the following to build Feast UI artifacts for use in `feast ui`
```
make build-ui
```
7. (Optional) install pixi. pixi is necessary to run step 8 for all python versions at once.
```sh
curl -fsSL https://pixi.sh/install.sh | bash
```
8. (Optional): Recompile python lock files. Only when you make changes to requirements or simply want to update python lock files to reflect latest versioons.
```sh
make lock-python-dependencies-all
```
9. Install development dependencies for Feast Python SDK and CLI. This will install package versions from the lock file, install editable version of feast and compile protobufs.

If running inside a virtual environment:
### Recompiling python lock files
Recompile python lock files. This only needs to be run when you make changes to requirements or simply want to update python lock files to reflect latest versions.

```sh
make install-python-ci-dependencies-uv-venv
make lock-python-dependencies-all
```

Otherwise:
### Building protos
```sh
make install-python-ci-dependencies-uv
make compile-protos-python
```

10. Spin up Docker Image
### Building a docker image for development
```sh
docker build -t docker-whale -f ./sdk/python/feast/infra/feature_servers/multicloud/Dockerfile .
```
Expand Down Expand Up @@ -405,7 +393,7 @@ It will:

### Testing with Github Actions workflows

Please refer to the maintainers [doc](maintainers.md) if you would like to locally test out the github actions workflow changes.
Please refer to the maintainers [doc](maintainers.md) if you would like to locally test out the github actions workflow changes.
This document will help you setup your fork to test the ci integration tests and other workflows without needing to make a pull request against feast-dev master.

## Feast Data Storage Format
Expand All @@ -414,4 +402,3 @@ Feast data storage contracts are documented in the following locations:

* [Feast Offline Storage Format](https://github.com/feast-dev/feast/blob/master/docs/specs/offline_store_format.md): Used by BigQuery, Snowflake \(Future\), Redshift \(Future\).
* [Feast Online Storage Format](https://github.com/feast-dev/feast/blob/master/docs/specs/online_store_format.md): Used by Redis, Google Datastore.

0 comments on commit b3b738f

Please sign in to comment.