Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change: Ci: cleaning up and restructure gh actions #1564

Merged
merged 3 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/install-openvas-dependencies.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ apt-get update && apt-get install --no-install-recommends --no-install-suggests
clang-tools \
cmake \
curl \
git \
lcov \
libgnutls28-dev \
libgpgme-dev \
Expand All @@ -28,6 +29,7 @@ apt-get update && apt-get install --no-install-recommends --no-install-suggests
libpopt0 \
libcurl4 \
libcurl4-gnutls-dev \
libhiredis0.14 \
&& rm -rf /var/lib/apt/lists/*

curl -L -o cgreen.tar.gz https://github.com/cgreen-devs/cgreen/archive/refs/tags/1.6.2.tar.gz -k
Expand Down
92 changes: 92 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Continuous Integration Workflow Documentation

This document outlines the Continuous Integration (CI) pipeline, detailing how to trigger releases and the specific roles of various jobs within the workflow.

## Release Trigger Process

To initiate a release, navigate to `Actions -> CI` in the GitHub repository, and click on `Run workflow`. Choose from the following options:
- `major`: For a major release with incompatible changes.
- `minor`: For a minor release introducing new features.
- `patch`: For a patch release focusing on bug fixes and minor improvements.
- `no_release`: To run the pipeline without releasing, updating the edge image.

## Jobs Overview

The CI pipeline incorporates multiple jobs, each with a specific function in the development lifecycle.

### 1. Initialization (`init`)

If the initialization fails it will prevent further execution of `build`.

- **Purpose**: Sets the release type based on the input or event that triggered the workflow.
- **Workflow File**: `init.yaml`

### 2. Unit Tests (`unittests`)
- **Purpose**: Executes unit tests to validate code changes.
- **Workflow File**: `tests.yml`

If the unit tests fails it will prevent further execution of `build`.

### 3. Build (`build`)
- **Purpose**: Compiles and builds the project, preparing it for testing and deployment.
- **Dependencies**: Requires successful completion of `unittests`.
- **Workflow File**: `build.yml`


If the build fails it will prevent further execution of `functional`.

### 4. Linting (`linting`)
- **Purpose**: Ensures code quality and consistency through linting.
- **Workflow File**: `linting.yml`

If linting fails it will not prevent execution of the other steps, as it may be that newer versions of the used tooling finds new linting issues that are not affecting the binary as much.


### 5. Functional Testing (`functional`)
- **Purpose**: Conducts functional tests on the build.
- **Dependencies**: Needs a successful `build`.
- **Workflow File**: `functional.yaml`

If the functional tests fail it will prevent further execution of `containerization`.

### 6. Containerization
- **Purpose**: Packages the build into Docker containers.
- **Jobs**:
- **Container**: Uses `push-container.yml`.
- **Container Testing**: Uses `push-container-testing.yml`.
- **Container Oldstable**: Uses `push-container-oldstable.yml`.
- **Dependencies**: Depends on `build`, `init`, and `functional`.

If the `containerization` fails the smoketests cannot be executed.

### 7. Smoke Tests (`smoketests`)
- **Purpose**: Conducts tests on helm chart based on the previously pushed docker image.
- **Conditions**: Excluded during pull request events.
- **Dependencies**: Relies on `container`, `build`, and `init`.
- **Workflow File**: `smoketest.yaml`

If the smoketests fail the helm chart will not be updated and releases be prevented.


### 8. Helm Chart Deployment (`helm`)
- **Purpose**: Deploys Helm chart, assuming `IMAGE_REGISTRY` is configured.
- **Conditions**: Triggered if `IMAGE_REGISTRY` is set.
- **Dependencies**: Depends on `smoketests`, `container`, `build`, and `init`.
- **Workflow File**: `push-helm-chart.yml`

### 9. Release (`release`)
- **Purpose**: Handles the release process for different version types.
- **Conditions**: Activated based on the release type set in `init`.
- **Dependencies**: Requires `smoketests`, `container`, `build`, and `init`.
- **Workflow File**: `release.yml`

## Secrets and Authentication

The CI workflow employs GitHub secrets for secure authentication and interaction with external services such as DockerHub.

### Utilized Secrets
- **DOCKERHUB_USERNAME**: DockerHub username.
- **DOCKERHUB_TOKEN**: Token for DockerHub with write access to the registry.
- **GREENBONE_BOT_TOKEN**: Token for Helm chart registry and GitHub repository operations.
- **GREENBONE_BOT**: Username for git commits.
- **GREENBONE_BOT_MAIL**: Email address for git commits.
66 changes: 0 additions & 66 deletions .github/workflows/build-container.yml

This file was deleted.

45 changes: 22 additions & 23 deletions .github/workflows/build-rust.yml → .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,24 @@
name: "rs-build"
name: "Build"

on: [workflow_call]

# This job builds the targets for x86_64 as well as aarch64. It is intented to
# be included in the other jobs by calling:
# ```
# jobs:
# name:
# uses: ./.github/workflows/build-rust.yml
# ```
#
# It saves the binaris:
# - nasl-cli
# - feed-verofier
#
# as rs-binaries.
#
jobs:
build:
C:
runs-on: ubuntu-latest
container: greenbone/gvm-libs:stable
steps:
- uses: actions/checkout@v4
- name: install dependencies
run: |
sh .github/install-openvas-dependencies.sh
- name: build
run: |
cmake -Bbuild -DCMAKE_C_COMPILER=/usr/share/clang/scan-build-14/libexec/ccc-analyzer
cmake --build build
Rust:
runs-on: "ubuntu-latest"
steps:
- uses: actions/checkout@v4
# create branch of version
- uses: actions/cache@v3
with:
path: |
Expand All @@ -47,12 +44,14 @@ jobs:
patchelf --replace-needed libz.so libz.so.1 target/aarch64-unknown-linux-gnu/release/nasl-cli
patchelf --replace-needed libz.so libz.so.1 target/x86_64-unknown-linux-gnu/release/nasl-cli
- run: mkdir assets/
- run: mv rust/target/aarch64-unknown-linux-gnu/release/openvasd assets/openvasd-aarch64-unknown-linux-gnu
- run: mv rust/target/x86_64-unknown-linux-gnu/release/openvasd assets/openvasd-x86_64-unknown-linux-gnu
- run: mv rust/target/aarch64-unknown-linux-gnu/release/nasl-cli assets/nasl-cli-aarch64-unknown-linux-gnu
- run: mv rust/target/x86_64-unknown-linux-gnu/release/nasl-cli assets/nasl-cli-x86_64-unknown-linux-gnu
- run: mv rust/target/aarch64-unknown-linux-gnu/release/feed-verifier assets/feed-verifier-aarch64-unknown-linux-gnu
- run: mv rust/target/x86_64-unknown-linux-gnu/release/feed-verifier assets/feed-verifier-x86_64-unknown-linux-gnu
- run: mkdir -p assets/linux/amd64
- run: mkdir -p assets/linux/arm64
- run: mv rust/target/aarch64-unknown-linux-gnu/release/openvasd assets/linux/arm64/openvasd
- run: mv rust/target/x86_64-unknown-linux-gnu/release/openvasd assets/linux/amd64/openvasd
- run: mv rust/target/aarch64-unknown-linux-gnu/release/nasl-cli assets/linux/arm64/nasl-cli
- run: mv rust/target/x86_64-unknown-linux-gnu/release/nasl-cli assets/linux/amd64/nasl-cli
- run: mv rust/target/aarch64-unknown-linux-gnu/release/feed-verifier assets/linux/arm64/feed-verifier
- run: mv rust/target/x86_64-unknown-linux-gnu/release/feed-verifier assets/linux/amd64/feed-verifier
- name: archive
uses: actions/upload-artifact@v3
with:
Expand Down
61 changes: 0 additions & 61 deletions .github/workflows/build_and_test.yml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: "CodeQL"

on:
push:
branches: [ main, oldstable, stable, middleware ]
branches: [ main ]
pull_request:
branches: [ main, oldstable, stable, middleware ]
branches: [ main ]
schedule:
- cron: '30 5 * * 0' # 5:30h on Sundays

Expand All @@ -16,12 +16,7 @@ jobs:
actions: read
contents: read
security-events: write
container: greenbone/gvm-libs:unstable

strategy:
fail-fast: false
matrix:
language: [ 'c' ]
container: greenbone/gvm-libs:edge

steps:
- name: Checkout repository
Expand All @@ -32,7 +27,7 @@ jobs:
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
languages: 'c'
# build between init and analyze ...
- name: Configure and Compile openvas-scanner
run: |
Expand Down
Loading
Loading