-
Notifications
You must be signed in to change notification settings - Fork 239
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
86a81bf
commit b0ed2c2
Showing
5 changed files
with
60 additions
and
50 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -34,10 +34,12 @@ The use of Docker in PEcAn is described in detail in the [PEcAn documentation](h | |
### Installing Docker | ||
|
||
To install Docker and docker-compose, see the docker documentation: | ||
|
||
- Docker Desktop in [MacOS](https://docs.docker.com/docker-for-mac/install/) or [Windows](https://docs.docker.com/docker-for-windows/install/) | ||
- Docker (e.g. [Ubuntu](https://docs.docker.com/compose/install/)) and [docker-compose](https://docs.docker.com/compose/install/) on your linux operating system. | ||
|
||
_Note for Linux users:_ add your user to the docker group. This will prevent you from having to use `sudo` to start the docker containers, and makes sure that any file that is written to a mounted volume is owned by you. This can be done using | ||
|
||
```sh | ||
# for linux users | ||
sudo adduser ${USER} docker. | ||
|
@@ -51,13 +53,13 @@ By default docker-compose will use the files `docker-compose.yml` and `docker-co | |
|
||
For Linux/MacOSX | ||
|
||
``` | ||
```sh | ||
cp docker-compose.dev.yml docker-compose.override.yml | ||
``` | ||
|
||
For Windows | ||
|
||
``` | ||
```sh | ||
copy docker-compose.dev.yml docker-compose.override.yml | ||
``` | ||
|
||
|
@@ -67,12 +69,12 @@ You can now use the command `docker-compose` to work with the containers setup f | |
|
||
The steps in this section only need to be done the first time you start working with the stack in docker. After this is done you can skip these steps. You can find more detail about the docker commands in the [pecan documentation](https://pecanproject.github.io/pecan-documentation/master/docker-index.html). | ||
|
||
* setup .env file | ||
* create folders to hold the data | ||
* load the postgresql database | ||
* load some test data | ||
* copy all R packages (optional but recommended) | ||
* setup for web folder development (optional) | ||
- setup .env file | ||
- create folders to hold the data | ||
- load the postgresql database | ||
- load some test data | ||
- copy all R packages (optional but recommended) | ||
- setup for web folder development (optional) | ||
|
||
#### .env file | ||
|
||
|
@@ -86,18 +88,18 @@ cp docker/env.example .env | |
|
||
For Windows | ||
|
||
``` | ||
```sh | ||
copy docker/env.example .env | ||
``` | ||
|
||
* `COMPOSE_PROJECT_NAME` set this to pecan, the prefix for all containers | ||
* `PECAN_VERSION` set this to develop, the docker image we start with | ||
- `COMPOSE_PROJECT_NAME` set this to pecan, the prefix for all containers | ||
- `PECAN_VERSION` set this to develop, the docker image we start with | ||
|
||
Both of these variables should also be uncommented by removing the # preceding them. At the end you should see the following if you run the following command `egrep -v '^(#|$)' .env`. If you have a windows system, you will need to set the variable PWD as well, and for linux you will need to set UID and GID (for rstudio). | ||
|
||
For Linux | ||
|
||
``` | ||
```sh | ||
echo "COMPOSE_PROJECT_NAME=pecan" >> .env | ||
echo "PECAN_VERSION=develop" >> .env | ||
echo "UID=$(id -u)" >> .env | ||
|
@@ -106,22 +108,22 @@ echo "GID=$(id -g)" >> .env | |
|
||
For MacOSX | ||
|
||
``` | ||
```sh | ||
echo "COMPOSE_PROJECT_NAME=pecan" >> .env | ||
echo "PECAN_VERSION=develop" >> .env | ||
``` | ||
|
||
For Windows: | ||
|
||
``` | ||
```sh | ||
echo "COMPOSE_PROJECT_NAME=pecan" >> .env | ||
echo "PECAN_VERSION=develop" >> .env | ||
echo "PWD=%CD%" >> .env | ||
``` | ||
|
||
Once you have setup `docker-compose.override.yml` and the `.env` files, it is time to pull all docker images that will be used. Doing this will make sure you have the latest version of those images on your local system. | ||
|
||
``` | ||
```sh | ||
docker-compose pull | ||
``` | ||
|
||
|
@@ -131,11 +133,10 @@ The goal of the development is to share the development folder with your contain | |
|
||
If you have uncommented the volumes in `docker-compose.override.yml` you will need to create the folders. Assuming you have not modified the values, you can do this with: | ||
|
||
``` | ||
```sh | ||
mkdir -p $HOME/volumes/pecan/{lib,pecan,portainer,postgres,rabbitmq,traefik} | ||
``` | ||
|
||
|
||
The following volumes are specified: | ||
|
||
- **pecan_home** : is the checked out folder of PEcAn. This is shared with the executor and rstudio container allowing you to share and compile PEcAn. (defaults to current folder) | ||
|
@@ -153,23 +154,22 @@ These folders will hold all the persistent data for each of the respective conta | |
|
||
First we bring up postgresql (we will start RabbitMQ as well since it takes some time to start): | ||
|
||
``` | ||
```sh | ||
docker-compose up -d postgres rabbitmq | ||
``` | ||
|
||
This will start postgresql and rabbitmq. We need to wait for a few minutes (you can look at the logs using `docker-compose logs postgres`) to see if it is ready. | ||
|
||
Once the database has finished starting up we will initialize the database. Now you can load the database using the following commands. The first command will make sure we have the latest version of the image, the second command will actually load the information into the database. | ||
|
||
``` | ||
```sh | ||
docker pull pecan/db | ||
docker run --rm --network pecan_pecan pecan/db | ||
``` | ||
|
||
|
||
Once that is done we create two users for BETY, first user is the guest user that you can use to login in the BETY interface. The second user is a user with admin rights. | ||
|
||
``` | ||
```sh | ||
docker-compose run --rm bety user guestuser guestuser "Guest User" [email protected] 4 4 | ||
docker-compose run --rm bety user carya illinois "Carya Demo User" [email protected] 1 1 | ||
``` | ||
|
@@ -178,7 +178,7 @@ docker-compose run --rm bety user carya illinois "Carya Demo User" carya@example | |
|
||
Once the database is loaded we can add some example data, some of the example runs and runs for the ED model, assume some of this data is available. This can take some time, but all the data needed will be copied to the `/data` folder in the pecan containers. As with the database we first pull the latest version of the image, and then execute the image to copy all the data: | ||
|
||
``` | ||
```sh | ||
docker pull pecan/data:develop | ||
docker run -ti --rm --network pecan_pecan --volume pecan_pecan:/data --env FQDN=docker pecan/data:develop | ||
``` | ||
|
@@ -206,26 +206,26 @@ docker run -ti --rm -v pecan_lib:/rlib pecan/base:develop cp -a /usr/local/lib/R | |
|
||
If you want to use the web interface, you will need to: | ||
|
||
1. Uncomment the web section from the `docker-compose.override.yml` file. This section includes three lines at the top of the file, just under the `services` section. Uncomment the lines that start `web:`, ` volumes:`, and `- pecan_web:`. | ||
1. Uncomment the web section from the `docker-compose.override.yml` file. This section includes three lines at the top of the file, just under the `services` section. Uncomment the lines that start `web:`, `volumes:`, and `- pecan_web:`. | ||
2. Then copy the config.php from the docker/web folder. You can do this using | ||
|
||
For Linux/MacOSX | ||
|
||
``` | ||
```sh | ||
cp docker/web/config.docker.php web/config.php | ||
``` | ||
|
||
For Windows | ||
|
||
``` | ||
```sh | ||
copy docker\web\config.docker.php web\config.php | ||
``` | ||
|
||
### PEcAn Development | ||
## PEcAn Development | ||
|
||
To begin development we first have to bring up the full PEcAn stack. This assumes you have done once the steps above. You don\'t need to stop any running containers, you can use the following command to start all containers. At this point you have PEcAn running in docker. | ||
|
||
``` | ||
```sh | ||
docker-compose up -d | ||
``` | ||
|
||
|
@@ -241,7 +241,7 @@ To compile the PEcAn code you can use the make command in either the rstudio con | |
|
||
You can submit your workflow either in the executor container or in rstudio container. For example to run the `docker.sipnet.xml` workflow located in the tests folder you can use: | ||
|
||
``` | ||
```sh | ||
docker-compose exec executor bash | ||
# inside the container | ||
cd /pecan/tests | ||
|
@@ -252,9 +252,9 @@ A better way of doing this is developed as part of GSOC, in which case you can l | |
|
||
# PEcAn URLs | ||
|
||
You can check the RabbitMQ server used by pecan using https://rabbitmq.pecan.localhost on the same server that the docker stack is running on. You can use rstudio either with http://server/rstudio or at http://rstudio.pecan.localhost. To check the traefik dashboard you can use http://traefik.pecan.localhost. | ||
You can check the RabbitMQ server used by pecan using <https://rabbitmq.pecan.localhost> on the same server that the docker stack is running on. You can use rstudio either with <http://server/rstudio> or at <http://rstudio.pecan.localhost>. To check the traefik dashboard you can use <http://traefik.pecan.localhost>. | ||
|
||
If the stack is running on a remote machine, you can use ssh and port forwarding to connect to the server. For example `ssh -L 8000:localhost:80` will allow you to use http://rabbitmq.pecan.localhost:8000/ in your browser to connect to the remote PEcAn server RabbitMQ. | ||
If the stack is running on a remote machine, you can use ssh and port forwarding to connect to the server. For example `ssh -L 8000:localhost:80` will allow you to use <http://rabbitmq.pecan.localhost:8000/> in your browser to connect to the remote PEcAn server RabbitMQ. | ||
|
||
# Directory Structure | ||
|
||
|
@@ -298,7 +298,7 @@ Small scripts that are used as part of the development and installation of PEcAn | |
|
||
If you want to start from scratch and remove all old data, but keep your pecan checked out folder, you can remove the folders where you have written the data (see `folders` below). You will also need to remove any of the docker managed volumes. To see all volumes you can do `docker volume ls -q -f name=pecan`. If you are sure, you can either remove them one by one, or remove them all at once using the command below. **THIS DESTROYS ALL DATA IN DOCKER MANAGED VOLUMES.**. | ||
|
||
``` | ||
```sh | ||
docker volume rm $(docker volume ls -q -f name=pecan) | ||
``` | ||
|
||
|
@@ -308,7 +308,7 @@ If you changed the docker-compose.override.yml file to point to a location on di | |
|
||
If you want to reset the pecan lib folder that is mounted across all machines, for example when there is a new version of PEcAn or a a new version of R, you will need to delete the volume pecan_lib, and repopulate it. To delete the volume use the following command, and then look at "copy R packages" to copy the data again. | ||
|
||
``` | ||
```sh | ||
docker-compose down | ||
docker volume rm pecan_lib | ||
``` | ||
|
@@ -323,27 +323,27 @@ This will leverage of NFS to mount the file system in your local docker image, c | |
|
||
First install nfs server: | ||
|
||
``` | ||
```sh | ||
apt-get install nfs-kernel-server | ||
``` | ||
|
||
Next export your home directory: | ||
|
||
``` | ||
```sh | ||
echo -e "$PWD\t127.0.0.1(rw,no_subtree_check,all_squash,anonuid=$(id -u),anongid=$(id -g))" | sudo tee -a /etc/exports | ||
``` | ||
|
||
And export the filesystem. | ||
|
||
``` | ||
```sh | ||
sudo exportfs -va | ||
``` | ||
|
||
At this point you have exported your home directory, only to your local machine. All files written to that exported filesystem will be owned by you (`id -u`) and your primary group (`id -g`). | ||
|
||
Finally we can modify the `docker-compose.override.yml` file to allow for writing files to your PEcAn folder as you: | ||
|
||
``` | ||
```sh | ||
volumes: | ||
pecan_home: | ||
driver_opts: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,3 @@ | ||
This folder contains components that either build on, leverage, or add capabilties to PEcAn. | ||
# README.md | ||
|
||
This folder contains components that either build on, leverage, or add capabilities to PEcAn |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
This folder contains published articles describing the development and application of PEcAn as well as tutorials. | ||
# Readme.md | ||
|
||
The full documentation can be found in the book_source directory, and is published at https://pecanproject.github.io/pecan-documentation/ with each new release. | ||
This folder contains published articles describing the development and application of PEcAn as well as tutorials. | ||
|
||
The full documentation can be found in the book_source directory, and is published at <https://pecanproject.github.io/pecan-documentation/> with each new release. |