Skip to content

Commit

Permalink
docs/DEVELOPMENT: document the CI
Browse files Browse the repository at this point in the history
Created a new "Continuous Integration (CI)" section in the developer's
guide to explain the e2e CI jobs types, how/who can execute them as well
as how to run some of them locally.

Fixes #113
Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
  • Loading branch information
wainersm committed Jul 10, 2023
1 parent 8c10735 commit 1b55caa
Showing 1 changed file with 59 additions and 0 deletions.
59 changes: 59 additions & 0 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,62 @@ Kind version `v0.16.0` have been successfully tested on the following Linux dist
- `Ubuntu 22.04`

>**Note**: Only `kata-clh` runtimeclass works with Kind cluster.
## Continuous Integration (CI)

In order to be merged your opened pull request (PR) should pass the static analysis checks and end-to-end (e2e) tests.

The e2e tests jobs are executed on a variety of CcRuntime, configurations and platforms. These jobs that require confidential hardware (Intel TDX, AMD SEV, IBM SE, etc...) run on bare-metal machines and are often referred as "TEE tests". The remaining tests (a.k.a "Non-TEE") are executed on Virtual Machines (VM) deployed on-demand.

The following jobs will check for regessions on the default CcRuntime:

|Job name | TEE | OS | VMM |
|---|---|---|---|
|cc-operator-e2e-ubuntu-20.04-s390x-containerd_kata-qemu | IBM SE | Ubuntu 20.04 (s390x) | QEMU |
|jenkins-ci-clh-tdx-x86_64 | Intel TDX | CentOS Stream 8 | Cloud Hypervisor |
|jenkins-ci-tdx-x86_64 | Intel TDX | CentOS Stream 8 | QEMU |
|tests-e2e-ubuntu-20.04-x86_64-containerd_kata-clh | Non-TEE | Ubuntu 20.04 | Cloud Hypervisor |
|tests-e2e-ubuntu-20.04-x86_64-containerd_kata-qemu | Non-TEE | Ubuntu 20.04 | QEMU |
|tests-e2e-ubuntu-20.04_sev-x86_64-containerd_kata-qemu-sev | AMD SEV | Ubuntu 20.04 | QEMU |
|tests-e2e-ubuntu-20.04_snp-x86_64-containerd_kata-qemu-snp | AMD SEV-SNP | Ubuntu 20.04 | QEMU |

>Note: jobs for Intel TDX run behind a firewall and, in general, the community has not access to their execution logs. If you hit a failure on any of those jobs then please request assistance from a member of Intel on our community. The easiest way to request help is to ask on our main Slack channel (#confidential-containers).
Additionally the following jobs will check regressions on the enclave-cc CcRuntime:

| Job name | TEE | OS |
|---|---|---|
|operator enclave-cc e2e tests| Intel SGX (Simulated Mode) | Ubuntu 22.04 |

Except for the enclave-cc jobs, the e2e are not triggered automatically. We recommend to trigger them only after some rounds of reviews to avoid wasting resources. They can be triggered only by writing `/test` in PR's comment.

>Note: only members with commit permission in the repository are allowed to trigger the e2e jobs. If you are not a committer then ask for help on our main Slack channel (#confidential-containers).
### Running e2e tests on your local machine

We recommend that you run the e2e Non-TEE tests on your local machine before opening a PR to check your changes will not break the CI so to avoid wasting resources. You can also use the approach describe below to debug and fix failures, or test changes on the scripts themselves.

The entry point script is [tests/e2e/run-local.sh](../tests/e2e/run-local.sh). It is going to install softwares and change the system's configuration, so we recommend that you run the e2e tests on VMs with nested virtualization support and a minimum of 8GB of memory, 50 GB of disk and 4 vCPUs.

Currently the e2e tests are supported on Ubuntu 20.04 or CentOS 8 Stream, and the only requirement is to have Ansible installed.

For example, to run on a fresh Ubuntu 20.04 VM:

```shell
sudo apt-get update -y
sudo apt-get install -y ansible python-is-python3
cd tests/e2e
export PATH="$PATH:/usr/local/bin"
./run-local.sh -r "kata-qemu"
```

Notice that the `-r` parameter passed to `run-local.sh` above specifies the runtimeClass to be tested. You can switch to, for example, `kata-clh` to test Cloud Hypervisor. Another useful parameter is `-u` which is used on bare-metal CI jobs to undo the changes at the execution end. See the script's help (`run-local.sh -h`) for further details and parameters.

### Running e2e test locally with Vagrant

Alternatively you can use [Vagrant](https://www.vagrantup.com) as we provide a [Vagrantfile](../tests/e2e/Vagrantfile) to automate the entire process: it will create the VM, push the local repository sources and finally execute `run-local.sh`. The same example above can be achieved by simply running:

```shell
export RUNTIMECLASS="kata-qemu"
vagrant up tests-e2e-ubuntu2004
```

0 comments on commit 1b55caa

Please sign in to comment.