-
Notifications
You must be signed in to change notification settings - Fork 0
Setting up the development environment for the EESSI bot
This page describes and illustrates the steps necessary to set up the development environment for the EESSI bot.
- GitHub account
- GitHub repository on whose events the bot shall react to
- Linux machine where the EESSI bot shall run (some steps may require sudo access)
We use smee.io as a service to relay events from GitHub to the EESSI bot. To do so, create a new channel on the page https://smee.io and note the URL, e.g., https://smee.io/CHANNEL_ID
On the Linux machine which runs the EESSI bot we need a tool which receives events relayed from https://smee.io/CHANNEL_ID and forwards it to the EESSI bot. We use the Smee client for this. The Smee client can be installed globally with
npm install -g smee-client
or per user
npm install smee-client
If you don't have npm on your system and don't have sudo access to easily install it, you may use a container as follows
mkdir smee
cd smee
singularity pull docker://node
singularity exec node_latest.sif npm install smee-client
cat << EOF > smee
#!/usr/bin/env bash
BASEDIR=$(dirname "$0")
singularity exec $BASEDIR/node_latest.sif $BASEDIR/node_modules/smee-client/bin/smee.js "$@"
EOF
chmod 700 smee
export PATH=$PATH:$PWD
Finally, run the Smee client as follows
smee --url https://smee.io/CHANNEL_ID
We first need to register an GitHub App, link it to the Smee.io channel, set a secret token to verify the webhook sender, set some permissions for the app, subscribe it to selected events and define that this app should only be installed in your account.
At the app settings page click "New GitHub App" and fill in the page, particular the following fields
- GitHub App name: give the app a name of you choice
- Homepage URL: use the Smee.io channel (https://smee.io/CHANNEL_ID) created in Step 1
- Webhook URL: use the Smee.io channel (https://smee.io/CHANNEL_ID) created in Step 1
- Webhook secret: create a secret token which is used to verify the webhook sender
- Permissions: assign permissions to the app it needs (e.g., read access to commits, issues, pull requests); those can be changed later on; some permissions (e.g., metadata will be selected automatically because of others you have chosen)
- Events: subscribe the app to events it shall react on (e.g., related to pull requests)
- Select that the app can only be installed by this (your) GitHub account
Click on "Create GitHub App"
The EESSI bot for the software layer is available from https://github.com/EESSI/eessi-bot-software-layer
Get the EESSI bot onto the Linux machine by running something like
git clone https://github.com/EESSI/eessi-bot-software-layer.git
If you want to develop the EESSI bot, it is recommended that you fork the repository and use the fork on the Linux machine.
The EESSI bot requires some Python packages to be installed. See https://github.com/EESSI/eessi-bot-software-layer#readme or simply run (the requirements.txt file is provided by the EESSI bot repository)
pip3 install --user -r requirements.txt
Troubles installing some of the requirements? You may try to install some of the dependencies by fixing their version. For example, on the CitC cluster (https://github.com/EESSI/hackathons/tree/main/2021-12/citc) installing PyGithub failed due to some problem installing its dependency PyNaCl. Apparently, PyGithub only required version 1.4.0 of PyNaCl but the most recent version 1.5.0 failed to install. Hence, when installing PyNaCl version 1.4.0 first, then PyGithub could be installed. Example commands
pip3 install --user PyNaCl==1.4.0
pip3 install --user -r requirements.txt
You need to set up three environment variables: GITHUB_TOKEN, GITHUB_APP_SECRET_TOKEN and GITHUB_PRIVATE_KEY.
Create a Personal Access Token (PAT) for your GitHub account via the page https://github.com/settings/tokens where you find a button "Generate new token". Give it meaningful name (field titled "Note") and set the expiration date. Then select the scopes this PAT will be used for. Then click "Generate token". On the result page, take note/copy the resulting token string -- it will only be shown once.
On the Linux machine set the environment variable GITHUB_TOKEN, e.g.
export GITHUB_TOKEN='THE_TOKEN_STRING'
The GitHub App Secret Token is used to verify the webhook sender. You should have created one already when registering a new GitHub App in Step 1.
On the Linux machine set the environment variable GITHUB_APP_SECTRET_TOKEN, e.g.
export GITHUB_APP_SECRET_TOKEN='THE_SECRET_TOKEN_STRING'
Note, depending on the characters used in the string you will likely have to use single quotes when setting the value of the environment variable.
The private key is needed to let the app authenticate when updating information at the repository such as commenting on PRs, adding labels, etc. You can create the key at the page of the GitHub App you have registered in Step 1.
Open the page https://github.com/settings/apps and then click on the icon left to the name of the GitHub App for the EESSI bot or the "Edit" button for the app. Near the end of the page you will find a section "Private keys" where you can create a private key by clicking on the button "Generate a private key". The private key should be automatically downloaded to your local computer. Copy it to the Linux machine and set the environment variable as follows
export GITHUB_PRIVATE_KEY="$(cat PATH_TO_PRIVATE_KEY_FILE)"
Change directory to eessi-bot-software-layer (which was created by cloning the repository in Step 3 - either the original one from EESSI or your fork). Then, simply run the bot by executing
./run.sh
Note, if you run the bot on a frontend of a cluster with multiple frontends make sure that both the Smee client and the bot run on the same machine.
The bot will log events into the console.
You need to install the GitHub App -- essentially telling GitHub to link the app to an account and one, several or all repositories on whose events the app then should act upon.
Go to the page https://github.com/settings/apps and select the app you want to install by clicking on the icon left to the apps name or on the "Edit" button right to the name of the app. On the next page you should see the menu item "Install App" on the left-hand side. When you click on this you should see a page with a list of accounts you can install the app on. Choose one and click on the "Install" button next to it. This leads to a page where you can select the repositories on whose the app should react to. Select one, multiple or all and click on the "Install" button.
Easiest test maybe a change -- creating a branch, committing a change, creating a pull request, etc -- to one of the repositories you have selected when installing the app in Step 5. Which events may be forwarded depends on to which events you have subscribed the app when you registered the app in Step 2.