From d2816eeee86a5b3bd97c1ea5c3ee4b1bd4c3ce21 Mon Sep 17 00:00:00 2001 From: Sarah Sloan Date: Wed, 10 Jan 2024 12:19:41 +0000 Subject: [PATCH] fsp-418 adding shared readmes --- readmes/README.md | 1 + readmes/python-repos-copilot.md | 16 ++++++ readmes/python-repos-db-development.md | 42 ++++++++++++++++ readmes/python-repos-ide-setup.md | 12 +++++ readmes/python-repos-paketo.md | 32 ++++++++++++ readmes/python-repos-setup.md | 68 ++++++++++++++++++++++++++ readmes/python-repos-testing.md | 13 +++++ 7 files changed, 184 insertions(+) create mode 100644 readmes/README.md create mode 100644 readmes/python-repos-copilot.md create mode 100644 readmes/python-repos-db-development.md create mode 100644 readmes/python-repos-ide-setup.md create mode 100644 readmes/python-repos-paketo.md create mode 100644 readmes/python-repos-setup.md create mode 100644 readmes/python-repos-testing.md diff --git a/readmes/README.md b/readmes/README.md new file mode 100644 index 00000000..380deb5c --- /dev/null +++ b/readmes/README.md @@ -0,0 +1 @@ +This folder contains shared markdown files that are linked to from other FSD Access Funding repositories so that common information, eg. how to install requirements is not duplicated. \ No newline at end of file diff --git a/readmes/python-repos-copilot.md b/readmes/python-repos-copilot.md new file mode 100644 index 00000000..42bd6b8a --- /dev/null +++ b/readmes/python-repos-copilot.md @@ -0,0 +1,16 @@ +Copilot is used for the deployment of the infrastructure configuration, which is all stored under the copilot folder in each service repository. The manifest files have been pre-generated by running through various initialisation steps that create the manifest files by prompting a series of questions, but do not _deploy_ the infrastructure. + +For each AWS account, these commands will need to be run _once_ to initialise the environment: + +`copilot app init pre-award` - this links the pre-award app with the current service, and associates the next commands with the service. Essentially, this provides context for the service to run under + +```bash + copilot init \ + --name \ + --app pre-award \ + --type 'Backend Service' \ + --image 'ghcr.io/communitiesuk/:latest' \ + --port 80 +``` + +This will initalise this service, using the current created image \ No newline at end of file diff --git a/readmes/python-repos-db-development.md b/readmes/python-repos-db-development.md new file mode 100644 index 00000000..95c5baec --- /dev/null +++ b/readmes/python-repos-db-development.md @@ -0,0 +1,42 @@ +## Local Database +If not using the docker runner (docker compose [setup](https://dluhcdigital.atlassian.net/wiki/spaces/FS/pages/79205102/Running+Access+Funding+Locally#Running-FSD-E2E-locally) for running all services together) you will need a local database instance for testing. This is particularly useful if making changes to models etc. + +1. Install Postgres +1. Create a database for the service you are using. Some repositories contain a script to create a DB for that service with any required extensions - see individual readmes for details. If the service you are using does not offer this, create a new DB manually: + ```bash + psql postgresql://localhost:5432 --user postgres + CREATE DATABASE ; + \l + ``` + Replace `` with the name of the database you wish to create, eg. `fsd_application_store`. The `\l` at the end lists all available databases, to confirm your new database has been created. +1. Create an environment variable to point at your local database, eg + + ```bash + # pragma: allowlist nextline secret + export DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:5432/ + ``` + +1. Run the service with `flask run` and it will connect to your local database as above + +## Development with SQLAlchemy +We use SQLAlchemy as our ORM implementation, and develop using a code-first approach. + +The below instructions assume you have already followed the [initial python setup guidelines](./python-repos-setup.md). + +Initialise the database for sql alchemy development: +```bash + flask db init +``` + +Then run any existing migrations to bring your local database up to date: +```bash + flask db upgrade +``` + +Whenever you make changes to database models, please run. +```bash + flask db migrate +``` +This updates the sql alchecmy migration files in `/db/migrations`. To see these updates reflected in your DB, you need to run `flask db upgrade` as above. + +Once your model changes are complete, commit and push these to github so that the migrations will be run in the pipelines to correctly upgrade the deployed db instances with your changes. \ No newline at end of file diff --git a/readmes/python-repos-ide-setup.md b/readmes/python-repos-ide-setup.md new file mode 100644 index 00000000..fc040f92 --- /dev/null +++ b/readmes/python-repos-ide-setup.md @@ -0,0 +1,12 @@ +# Pre-Commit +Each repo contains a .pre-commit-config.yaml which all developers should use to check styles etc before pushing changes. + +Run the following while in your virtual enviroment: + +```bash + pip install pre-commit black + + pre-commit install +``` + +Once the above is done you will have autoformatting and pep8 compliance built into your workflow. You will be notified of any pep8 errors during commits. \ No newline at end of file diff --git a/readmes/python-repos-paketo.md b/readmes/python-repos-paketo.md new file mode 100644 index 00000000..f6266722 --- /dev/null +++ b/readmes/python-repos-paketo.md @@ -0,0 +1,32 @@ +# Build with Paketo + +[Pack](https://buildpacks.io/docs/tools/pack/cli/pack_build/) + +[Paketo buildpacks](https://paketo.io/) + +```bash + pack build --builder paketobuildpacks/builder:base +``` + +Example: + + [~/work/repos/funding-service-design-fund-store] pack build paketo-demofsd-app --builder paketobuildpacks/builder:base + *** + Successfully built image paketo-demofsd-app + + +You can then use that image with docker to run a container + +```bash + docker run -d -p 8080:8080 --env PORT=8080 --env FLASK_ENV=dev [envs] paketo-demofsd-app +``` + +`envs` needs to include values for each of: +SENTRY_DSN +GITHUB_SHA + +```bash + docker ps -a + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + 42633142c619 paketo-demofsd-app "/cnb/process/web" 8 seconds ago Up 7 seconds 0.0.0.0:8080->8080/tcp peaceful_knuth +``` \ No newline at end of file diff --git a/readmes/python-repos-setup.md b/readmes/python-repos-setup.md new file mode 100644 index 00000000..6cf39be7 --- /dev/null +++ b/readmes/python-repos-setup.md @@ -0,0 +1,68 @@ +# Setup steps for FSD Access Funding Python Repositories + +## Prerequisites +- python == 3.10 + +## Installation + +Clone the repository - scripts to clone all access funding repositories available [here](https://dluhcdigital.atlassian.net/wiki/spaces/FS/pages/79205102/Running+Access+Funding+Locally#Cloning-the-Repos) + +### Create a Virtual environment + +```bash + python3 -m venv .venv +``` + +### Enter the virtual environment + +...either macOS using bash: + +```bash + source .venv/bin/activate +``` + +...or if on Windows using Command Prompt: + +```bash + .venv\Scripts\activate.bat +``` + +### Install dependencies +From the top-level directory enter the command to install pip and the dependencies of the project + +```bash + python3 -m pip install --upgrade pip && pip install -r requirements-dev.txt +``` +NOTE: requirements-dev.txt and requirements.txt are updated using [pip-tools pip-compile](https://github.com/jazzband/pip-tools) +To update requirements please manually add the dependencies in the .in files (not the requirements.txt files) +Then run: + +```bash + pip-compile requirements.in + + pip-compile requirements-dev.in +``` + +## How to use +To run the application standalone, enter the virtual environment as described above, then: + +```bash + flask run +``` + +Note: Not all applications will function correctly without other dependencies also being avaialble, eg. Databases, other microservices. To run all the microservices together, we have a `docker-compose` file that links them, documented [here](https://dluhcdigital.atlassian.net/wiki/spaces/FS/pages/79205102/Running+Access+Funding+Locally#Running-FSD-E2E-locally) + +## Run with Gunicorn + +In deployed environments the service is run with gunicorn. You can run the service locally with gunicorn to test + +First set the FLASK_ENV environment you wish to test eg: + +```bash + export FLASK_ENV=dev +``` +Then run gunicorn using the following command: + +```bash + gunicorn wsgi:app -c run/gunicorn/local.py +``` \ No newline at end of file diff --git a/readmes/python-repos-testing.md b/readmes/python-repos-testing.md new file mode 100644 index 00000000..df17c259 --- /dev/null +++ b/readmes/python-repos-testing.md @@ -0,0 +1,13 @@ +# Unit Testing + +We write unit tests using Pytest. + +To run all tests in a development environment run: +```bash + pytest +``` + +## Pytest Fixtures +Fixtures shared across a single project are contained in `conftest.py`. + +fsd_utils provides some common fixtures for dealing with database testing, see [FSD Utils README](https://github.com/communitiesuk/funding-service-design-utils?tab=readme-ov-file#fixtures) \ No newline at end of file