Cleanair Data package docs

containers/cleanair/cleanair_data/dockerfiles/cleanair_base.Dockerfile

@@ -0,0 +1,33 @@
+# Use an official Python runtime as a parent image
+FROM python:3.10
+
+# Update certificates
+RUN apt update \
+    && apt install -y \
+    build-essential \
+    ca-certificates \
+    cmake \
+    gfortran \
+    libgeos-dev
+
+# download eccodes for ECMWF satellite
+WORKDIR /app
+ENV ECCODES_MAJOR_VERSION=2
+ENV ECCODES_MINOR_VERSION=26
+ENV ECCODES_PATCH_VERSION=0
+ENV ECCODES_VERSION=${ECCODES_MAJOR_VERSION}.${ECCODES_MINOR_VERSION}.${ECCODES_PATCH_VERSION}
+ENV ECCODES_SRC_DIR=eccodes-${ECCODES_VERSION}-Source
+ENV ECCODES_TAR_NAME=eccodes-${ECCODES_VERSION}-Source.tar.gz
+RUN wget -c https://confluence.ecmwf.int/download/attachments/45757960/${ECCODES_TAR_NAME}
+RUN tar -xzf ${ECCODES_TAR_NAME}
+
+# install eccodes with CMake
+RUN mkdir ${ECCODES_SRC_DIR}/build
+WORKDIR ${ECCODES_SRC_DIR}/build
+RUN cmake -DENABLE_AEC=OFF ..
+RUN cmake --build .
+RUN cmake --install .
+
+# remove eccodes build directory
+WORKDIR /app
+RUN rm -r ${ECCODES_SRC_DIR}/build

containers/cleanair/cleanair_data/docs/README.md

@@ -1,4 +1,45 @@
-# Inistalation
+# London Air Quality Research Project
 
-conda create -n urbanairdata
-conda activate urbanairdata
+This README provides an overview of the data package for the London Air Quality project. The data package includes essential documentation and resources to help you get started with the installation, development, and usage of the data associated with this project.
+
+## Contents
+
+- [📥 Installation](installation.md)
+- [👨‍💻 Developer Guide](developer.md)
+- [🐳 Docker Guide](docker.md)
+- [📊 Datasets](datasets.md)
+- [🔐 Secret Files](secretfile.md)
+- [📋 Cheat Sheet](cheat.md)
+- [🙌 Contributors](contributors.md)
+
+## Overview
+
+The London Air Quality data package provides access to comprehensive datasets and tools to facilitate the analysis and visualization of air quality in London. This package is designed to support researchers, developers, and analysts working on air quality projects.
+
+## 📥 Installation
+
+For detailed instructions on how to install the data package, please refer to the [Installation Guide](docs/installation.md).
+
+## 👨‍💻 Developer Guide
+
+To contribute to the development of this data package or to understand the underlying architecture, refer to the [Developer Guide](docs/developer.md).
+
+## 🐳 Docker Guide
+
+This project includes Docker support for creating consistent and reproducible development environments. For more information, see the [Docker Guide](docs/docker.md).
+
+## 📊 Datasets
+
+Comprehensive details about the datasets included in this project are available in the [Datasets Guide](docs/datasets.md).
+
+## 🔐 Secret Files
+
+To learn about managing and accessing secret files required by the project, refer to the [Secret Files Guide](docs/secretfile.md).
+
+## 📋 Cheat Sheet
+
+A quick reference guide for common commands and configurations can be found in the [Cheat Sheet](docs/cheat.md).
+
+## 🙌 Contributors
+
+We acknowledge and appreciate the contributions from various individuals and organizations. For a list of contributors, see the [Contributors Guide](docs/contributors.md).

containers/cleanair/cleanair_data/docs/codebase.md

@@ -0,0 +1 @@
+# Codebase

containers/cleanair/cleanair_data/docs/configure_local_database.md

@@ -0,0 +1,130 @@
+## Installing and Starting PostgreSQL
+
+To install and start PostgreSQL on macOS, you can use Homebrew Services. Below are detailed, step-by-step instructions.
+
+### Step-by-Step Instructions
+
+1. **Install Homebrew:**
+   If Homebrew is not already installed on your system, open your terminal and execute the following command:
+
+   ```bash
+   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
+   ```
+
+   **Documentation:**
+   - [Homebrew Installation](https://brew.sh/)
+
+2. **Install PostgreSQL using Homebrew:**
+   With Homebrew installed, install PostgreSQL by running:
+
+   ```bash
+   brew install postgresql
+   ```
+
+   This command installs the latest version of PostgreSQL available via Homebrew.
+
+   **Documentation:**
+   - [PostgreSQL Homebrew Installation](https://formulae.brew.sh/formula/postgresql)
+
+3. **Install Homebrew Services:**
+   Homebrew Services is an extension for Homebrew that simplifies the process of starting and stopping services. Install it with:
+
+   ```bash
+   brew tap homebrew/services
+   ```
+
+   **Documentation:**
+   - [Homebrew Services](https://github.com/Homebrew/homebrew-services)
+
+4. **Start PostgreSQL Service:**
+   Start PostgreSQL as a service using Homebrew Services:
+
+   ```bash
+   brew services start postgresql
+   ```
+
+   This command runs PostgreSQL as a background service, which will automatically restart after a reboot.
+
+   **Documentation:**
+   - [Starting Services with Homebrew](https://github.com/Homebrew/homebrew-services#readme)
+
+### If You Installed PostgreSQL Using Conda
+
+To set up the PostgreSQL server and users, follow these steps:
+
+1. **Initialize the Database:**
+
+   ```bash
+   initdb -D mylocal_db
+   ```
+
+2. **Start the PostgreSQL Server:**
+
+   ```bash
+   pg_ctl -D mylocal_db -l logfile start
+   ```
+
+3. **Create a Database:**
+
+   ```bash
+   createdb --owner=${USER} myinner_db
+   ```
+
+4. **Start the Server for Future Sessions:**
+   When you want to work in this environment again, start the server with:
+
+   ```bash
+   pg_ctl -D mylocal_db -l logfile start
+   ```
+
+5. **Stop the Server:**
+   You can stop the server with:
+
+   ```bash
+   pg_ctl -D mylocal_db stop
+   ```
+
+## Creating a Local Secrets File
+
+Refer to the [secret file documentation](secretfile.md) for detailed instructions.
+
+In some cases, your default username may be your OS user. Adjust the username in the secrets file if necessary.
+
+## Create a Database on Your Machine
+
+Create a database named `cleanair_test_db` with the following command:
+
+```bash
+createdb cleanair_test_db
+```
+
+## Create Schema and Roles
+
+To set up the database schema and create roles, follow these steps:
+
+1. **Set the Location of Your Secrets File:**
+
+   ```bash
+   export DB_SECRET_FILE=$(pwd)/.secrets/.db_secrets_offline.json
+   ```
+
+2. **Configure Database Roles:**
+   Run the following command to configure database roles:
+
+   ```bash
+   python containers/entrypoints/setup/configure_db_roles.py -s $DB_SECRET_FILE -c configuration/database_role_config/local_database_config.yaml
+   ```
+
+### Insert Static Data
+
+**TODO:** Instructions for inserting static data into the server database need to be provided.
+
+### Check the Database Configuration
+
+Verify that everything is configured correctly by running:
+
+```bash
+pytest containers/tests/test_database_init --secretfile $DB_SECRET_FILE
+```
+
+By following these steps, you will have PostgreSQL installed, configured, and running on your macOS system.

containers/cleanair/cleanair_data/docs/contributors.md

@@ -0,0 +1,22 @@
+## Active contributors
+
+| Name               | GitHub ID                                            | Email                       | Admin  |
+| ------------------ | -----------------------------------------------------| --------------------------- | ------ |
+| Sueda Ciftci       | [@edasu](https://github.com/edasu)                   | <[email protected]> | Infrastructure, Software Engineering |
+| Oliver Hamelijnck  | [@defaultobject](https://github.com/defaultobject)   | <[email protected]>  | Modelling |
+| Patrick O'Hara     | [@PatrickOHara](https://github.com/PatrickOHara)     | <[email protected]>       | Validation, visualisation |
+
+## Alumni contributors
+
+| Name               | GitHub ID                                            | Email                       | Admin  |
+| ------------------ | -----------------------------------------------------| --------------------------- | ------ |
+| James Brandreth    | [@jamesbrandreth](https://github.com/jamesbrandreth) | <[email protected]> / <[email protected]> | Infrastructure, Odysseus |
+| Oscar Giles        | [@OscartGiles](https://github.com/OscartGiles)       | <[email protected]>       | Infrastructure, Prod Database, Kubernetes Cluster |
+| Chance Haycock     | [@chancehaycock](https://github.com/chancehaycock)   | <[email protected]>     | |
+| Christy Nakou      | [@ChristyNou](https://github.com/ChristyNou)         | <[email protected]>       | |
+| Harry Moss     | [@harryjmoss](https://github.com/harryjmoss)     | <[email protected]>       | |
+| David Perez-Suarez | [@dpshelio](https://github.com/dpshelio)             | <[email protected]>  | |
+| James Robinson     | [@jemrobinson](https://github.com/jemrobinson)       | <[email protected]>    | Infrastructure, Prod Database, Kubernetes Cluster |
+| Tim Spain          | [@timspainUCL](https://github.com/timspainUCL)       | <[email protected]>         | |
+| Edward Thorpe-Woods | [@TeddyTW](https://github.com/TeddyTW)              | <[email protected]>| |
+| James Walsh | [@dead-water](https://github.com/dead-water)              | <[email protected]>| Infrastructure, Odysseus |