Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify workflows #661

Merged
merged 12 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 0 additions & 25 deletions .github/actions/build-rpch/action.yml

This file was deleted.

26 changes: 26 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# GitHub Workflows

This document describes the GitHub workflows used in this project.

## Build

- Builds, Lint, Test
- Publishes the docker images with version `x.y.z-pr.<PR_NUMBER>`
- If the PR has attached the label `deploy_staging` it will update the docker images `staging` and will deploy the version in the staging environment

## Merge PR

- Publishes the docker image with `latest`
- Publishes the docker image with `staging`
- Deploy the tag `staging` in the staging environment

## Close release

This is a workflow triggered manually from Github Actions [Close Release](https://github.com/hoprnet/RPCh/actions/workflows/release.yaml). The tasks performed by this workflow include:

- Publishes the docker image with `x.y.z`
- Create a Github release
- Tag code
- Changelog is autogenerated by commit linear history of the release
- Bumps the new version
- Sends a Zulip notification
123 changes: 123 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
name: Build

on:
pull_request:
types:
- opened
- synchronize
- reopened
- labeled

concurrency:
group: build
cancel-in-progress: true

jobs:
build:
name: Build
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18.x, 20.x, 22.x]

services:
# required by unit tests
postgres:
image: postgres
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
# set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup Node.js
uses: hoprnet/hopr-workflows/actions/setup-node-js@master
with:
node-version: ${{ matrix.node-version }}

- name: Building
run: yarn build

- name: Linting
run: yarn lint:ci

- name: Formatting
run: yarn format:ci

- name: Review dependencies
# TODO: update ethers to v6 in RPCh SDK to remove the ignore-path
run: yarn depcheck --ignore-path="examples"

- name: Testing
run: yarn test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/postgres

# - name: Run E2E tests
# run: yarn test:e2e

publish:
name: Publish
runs-on: self-hosted-hoprnet-small
needs: build
strategy:
matrix:
project:
- discovery-platform
- rpc-server
- availability-monitor
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup GCP
id: gcp
uses: hoprnet/hopr-workflows/actions/setup-gcp@master
with:
google-credentials: ${{ secrets.GOOGLE_HOPRASSOCIATION_CREDENTIALS_REGISTRY }}
login-artifact-registry: 'true'
install-sdk: 'true'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: kubernetes

- name: Get PR version
id: variables
run: |
ausias-armesto marked this conversation as resolved.
Show resolved Hide resolved
PR_VERSION=$(jq -r '.version' apps/"${{ matrix.project }}"/package.json)-pr."${{ github.event.pull_request.number }}"
echo "PR_VERSION=${PR_VERSION}" >> $GITHUB_OUTPUT
Comment on lines +99 to +102
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Improve PR version extraction for better security.

The current method of extracting the PR version could be improved to prevent potential command injection.

Consider using jq with the --raw-output (-r) option and proper quoting:

- PR_VERSION=$(jq -r '.version' apps/"${{ matrix.project }}"/package.json)-pr."${{ github.event.pull_request.number }}"
+ PR_VERSION="$(jq --raw-output '.version' "apps/${{ matrix.project }}/package.json")-pr.${{ github.event.pull_request.number }}"

This change ensures proper quoting and uses the full --raw-output option for clarity.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
id: variables
run: |
PR_VERSION=$(jq -r '.version' apps/"${{ matrix.project }}"/package.json)-pr."${{ github.event.pull_request.number }}"
echo "PR_VERSION=${PR_VERSION}" >> $GITHUB_OUTPUT
id: variables
run: |
PR_VERSION="$(jq --raw-output '.version' "apps/${{ matrix.project }}/package.json")-pr.${{ github.event.pull_request.number }}"
echo "PR_VERSION=${PR_VERSION}" >> $GITHUB_OUTPUT
Tools
actionlint

100-100: shellcheck reported issue in this script: SC2086:info:2:36: Double quote to prevent globbing and word splitting

(shellcheck)


- name: Build and push docker image
uses: docker/build-push-action@v6
with:
push: true
file: "./apps/${{ matrix.project }}/Dockerfile"
tags: ${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ matrix.project }}:${{ steps.variables.outputs.PR_VERSION }}

- name: Tag staging
if: contains(github.event.pull_request.labels.*.name, 'deploy_staging') || github.event.label.name == 'deploy_staging' && github.event.action == 'labeled'
run: |
gcloud artifacts docker tags add ${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ matrix.project }}:${{ steps.variables.outputs.PR_VERSION }} ${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ matrix.project }}:staging

deploy:
name: Deploy
needs: publish
if: contains(github.event.pull_request.labels.*.name, 'deploy_staging') || github.event.label.name == 'deploy_staging' && github.event.action == 'labeled'
uses: ./.github/workflows/deploy.yaml
with:
branch: ${{ github.event.pull_request.head.ref }}
secrets: inherit
38 changes: 38 additions & 0 deletions .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
name: Deploy

on:
workflow_call:
inputs:
branch:
required: true
type: string

concurrency:
group: deploy
cancel-in-progress: true

jobs:

deploy:
name: Deploy staging
runs-on: self-hosted-hoprnet-small
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
ref: ${{ inputs.branch }}

- name: Setup GCP
id: gcp
uses: hoprnet/hopr-workflows/actions/setup-gcp@master
with:
google-credentials: ${{ secrets.GCP_SA_TERRAFORM_JSON_STAGING }}
project: hopr-staging
login-gke: 'true'
ausias-armesto marked this conversation as resolved.
Show resolved Hide resolved

- name: "Restart deployment"
run: |
echo "[INFO] Restarting degen deployment"
kubectl rollout restart deployments -n uhttp availability-monitor discovery-platform
kubectl rollout status -w deployments -n uhttp availability-monitor discovery-platform
ausias-armesto marked this conversation as resolved.
Show resolved Hide resolved
57 changes: 57 additions & 0 deletions .github/workflows/merge.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Merge PR

on:
pull_request:
types:
- closed
branches:
- main

concurrency:
group: merge
cancel-in-progress: false

jobs:
merge:
name: Merge PR
runs-on: self-hosted-hoprnet-small
if: github.event.pull_request.merged == true
strategy:
matrix:
project:
- discovery-platform
- rpc-server
- availability-monitor

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup GCP
id: gcp
uses: hoprnet/hopr-workflows/actions/setup-gcp@master
with:
google-credentials: ${{ secrets.GOOGLE_HOPRASSOCIATION_CREDENTIALS_REGISTRY }}
login-artifact-registry: 'true'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: kubernetes

- name: Build and push docker image
uses: docker/build-push-action@v6
with:
push: true
file: "./apps/${{ matrix.project }}/Dockerfile"
tags: |
${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ matrix.project }}:staging
${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ matrix.project }}:latest

deploy:
name: Deploy staging
needs: merge
uses: ./.github/workflows/deploy.yaml
with:
branch: ${{ github.event.pull_request.head.ref }}
secrets: inherit
110 changes: 110 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: Close release

on:
workflow_dispatch:
inputs:
release_type:
description: 'Next version type'
required: true
type: choice
default: 'patch'
options:
- patch
- minor
- major
project:
description: 'Project'
required: true
type: choice
options:
- discovery-platform
- rpc-server
- availability-monitor

concurrency:
group: release
cancel-in-progress: false

jobs:
release:
name: Close release
runs-on: self-hosted-hoprnet-small

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup Node.js
uses: hoprnet/hopr-workflows/actions/setup-node-js@master
with:
node-version: ${{ vars.NODE_VERSION }}

- name: Setup GCP
id: gcp
uses: hoprnet/hopr-workflows/actions/setup-gcp@master
with:
google-credentials: ${{ secrets.GOOGLE_HOPRASSOCIATION_CREDENTIALS_REGISTRY }}
login-artifact-registry: 'true'

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: kubernetes

- name: Building
run: yarn build

- name: Linting
run: yarn lint:ci

- name: Formatting
run: yarn format:ci

- name: Testing
run: yarn test
ausias-armesto marked this conversation as resolved.
Show resolved Hide resolved

- name: Setup environment variables
id: environment
run: |
PACKAGE_VERSION=$(jq -r '.version' "apps/${{ inputs.project }}/package.json")
echo "release_version=${PACKAGE_VERSION}" >> $GITHUB_OUTPUT

- name: Create Release
uses: softprops/action-gh-release@v2
with:
name: '${{ inputs.project }} - v${{ steps.environment.outputs.release_version }}'
tag_name: v${{ steps.environment.outputs.release_version }}

- name: Build and push docker image
uses: docker/build-push-action@v6
with:
push: true
file: "./apps/${{ inputs.project }}/Dockerfile"
tags: ${{ vars.DOCKER_IMAGE_REGISTRY }}/${{ inputs.project }}:${{ steps.environment.outputs.release_version }}
Comment on lines +66 to +83
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

LGTM: Well-implemented release creation and Docker image push

The steps for creating a GitHub release and pushing a Docker image are well-structured and use appropriate actions.

The static analysis tool flagged a potential issue with quoting in the jq command. To address this, please update line 69 as follows:

- PACKAGE_VERSION=$(jq -r '.version' "apps/${{ inputs.project }}/package.json")
+ PACKAGE_VERSION=$(jq -r '.version' "apps/${{ inputs.project }}/package.json")

This change ensures proper quoting and prevents potential issues with word splitting and globbing.

Committable suggestion was skipped due to low confidence.

Tools
actionlint

68-68: shellcheck reported issue in this script: SC2086:info:2:46: Double quote to prevent globbing and word splitting

(shellcheck)


- name: Bump Version
id: bump
run: |
npm version ${{ inputs.release_type }} -w "apps/${{ inputs.project }}" --no-git-tag-version
BUMP_VERSION=$(jq -r '.version' "apps/${{ inputs.project }}/package.json")
echo "bump_version=${BUMP_VERSION}" >> $GITHUB_OUTPUT

- uses: EndBug/add-and-commit@v9
with:
add: 'apps/${{ inputs.project}}'
default_author: github_actor
new_branch: main
message: 'Bump ${{ inputs.project }} version to ${{ steps.bump.outputs.bump_version }}'
pathspec_error_handling: exitImmediately

- name: Notify new release
uses: zulip/github-actions-zulip/send-message@v1
with:
api-key: ${{ secrets.ZULIP_API_KEY }}
email: ${{ secrets.ZULIP_EMAIL }}
organization-url: 'https://hopr.zulipchat.com'
type: 'stream'
to: 'Releases'
topic: 'main'
content: |
I'm thrilled to inform the new **${{ vars.DOCKER_IMAGE_NAME }}** version **${{ steps.environment.outputs.release_version }}** has been released.
ausias-armesto marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading