-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add CI workflows #357
Merged
Merged
Add CI workflows #357
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# CI workflows | ||
|
||
This directory contains workflows that are run for Continuous Integration (CI) testing. | ||
|
||
## Static code checks | ||
|
||
The workflow that runs static code checks is based on the clang-format and clang-tidy requirements from CMSSW. They are run automatically for commits in pull-requests, and should be roughly equivalent to running `scram build code-checks` and `scram build code-format`. They are fairly lenient, checking only the lines that were changed without making any modifications. The aim is to make sure that the code is compliant for the CMSSW integration. More information can be found [here](https://cms-sw.github.io/PRWorkflow.html). As work towards the full integration with CMSSW progresses, we will tune these checks to become more stringent and closer to what is done during the CMSSW CI checks. | ||
|
||
## Testing worflows | ||
|
||
The workflows that run the code and produce validation and comparison plots for the standalone and CMSSW setups depend on custom GitHub actions that are located in [SegmentLinking/TrackLooper-actions](https://github.com/SegmentLinking/TrackLooper-actions). Most of the workflow is offloaded to these actions, so it is easy to adjust the process without modifying this repository. These workflows are much more time-consuming, so they must be manually triggered by leaving a comment on the PR containing `/run standalone` and/or `/run cmssw`. | ||
|
||
When testing the CMSSW integration, a PR in this repository might depend on a corresponding PR in the [CMSSW fork](https://github.com/SegmentLinking/cmssw). Since the two PRs need to be tested together, using `/run cmssw` would not work. For this reason, there is an optional parameter to specify a branch of the CMSSW repository, `/run cmssw some-branch`. This command needs to be in its own line in the comment or it will not work correctly. | ||
|
||
## First-time configuration | ||
|
||
There is a one-time configuration step that needs to be done in order to allow the CI to upload the plots to the archival repository. We leave the instructions here in case it ever needs to be done again. | ||
|
||
1. Generate a new SSH key with `ssh-keygen -t ed25519`. Save it as `tmp_deploy_key`, and leave it without a password. | ||
1. Create a new repository secret named `DEPLOY_PRIVATE_KEY` and set it to the contents of `tmp_deploy_key`. | ||
1. In the repository that will store the validation plots, add a new deployment key with the contents of `tmp_deploy_key.pub`. | ||
1. Delete both `tmp_deploy_key` and `tmp_deploy_key.pub` to prevent security issues. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
name: Static checks | ||
|
||
on: | ||
pull_request: | ||
branches: master | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.event.issue.number }} | ||
cancel-in-progress: true | ||
|
||
permissions: | ||
contents: read | ||
pull-requests: write | ||
|
||
jobs: | ||
linter: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout PR | ||
uses: actions/checkout@v4 | ||
- name: Mount cvmfs | ||
uses: cvmfs-contrib/github-action-cvmfs@v3 | ||
- name: Gather paths of needed headers | ||
shell: bash | ||
run: | | ||
source setup.sh | ||
INCLUDE_FLAGS="-I$TRACKLOOPERDIR -I$BOOST_ROOT/include -I$ALPAKA_ROOT/include -I$CUDA_HOME/include -I$ROOT_ROOT/include -I$CMSSW_BASE/src" | ||
INCLUDE_FLAGS="$INCLUDE_FLAGS -I/cvmfs/cms.cern.ch/el8_amd64_gcc11/external/gcc/11.4.1-30ebdc301ebd200f2ae0e3d880258e65/include/c++/11.4.1" | ||
INCLUDE_FLAGS="$INCLUDE_FLAGS -I/cvmfs/cms.cern.ch/el8_amd64_gcc11/external/gcc/11.4.1-30ebdc301ebd200f2ae0e3d880258e65/include/c++/11.4.1/x86_64-redhat-linux-gnu/" | ||
INCLUDE_FLAGS="$INCLUDE_FLAGS -I/cvmfs/cms.cern.ch/el8_amd64_gcc11/external/gcc/11.4.1-30ebdc301ebd200f2ae0e3d880258e65/lib/gcc/x86_64-redhat-linux-gnu/11.4.1/include" | ||
echo "include-flags=$INCLUDE_FLAGS" >> $GITHUB_ENV | ||
# Note: Ideally we would use compiledb to generate a compile_commands.json file, but I couldn't get it to work properly with clang-tidy. | ||
# This alternative of simply passing some extra arguments to clang-tidy seems to work fine (at least for now). | ||
- name: Run linter | ||
uses: cpp-linter/cpp-linter-action@v2 | ||
id: linter | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
with: | ||
style: file | ||
thread-comments: true | ||
# only use checks in .clang-tidy file | ||
tidy-checks: '' | ||
lines-changed-only: true | ||
ignore: .github|bin|code|data|efficiency | ||
extra-args: '-ferror-limit=0 -std=c++17 -DALPAKA_ACC_CPU_B_SEQ_T_SEQ_ENABLED ${{ env.include-flags }}' | ||
- name: Fail job if there were checks that failed | ||
if: steps.linter.outputs.checks-failed > 0 | ||
run: exit 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,131 @@ | ||
name: Testing | ||
|
||
on: | ||
issue_comment: | ||
types: [created] | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.event.issue.number }} | ||
cancel-in-progress: true | ||
|
||
permissions: | ||
contents: read | ||
pull-requests: write | ||
|
||
jobs: | ||
standalone: | ||
if: > | ||
github.event.issue.pull_request && | ||
contains(github.event.comment.body, '/run standalone') && | ||
(github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'CONTRIBUTOR') | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout PR | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: refs/pull/${{ github.event.issue.number }}/head | ||
- name: Build and run PR | ||
id: build-and-run | ||
uses: SegmentLinking/TrackLooper-actions/standalone@v1 | ||
with: | ||
pr-number: ${{ github.event.issue.number }} | ||
- name: Upload plots to archival repo | ||
uses: s0/git-publish-subdir-action@develop | ||
env: | ||
REPO: [email protected]:${{ steps.build-and-run.outputs.archive-repo }}.git | ||
BRANCH: ${{ steps.build-and-run.outputs.archive-branch }} | ||
FOLDER: ${{ steps.build-and-run.outputs.archive-dir }} | ||
SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_PRIVATE_KEY }} | ||
TARGET_DIR: ${{ steps.build-and-run.outputs.archive-dir }} | ||
- name: Comment on PR | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
github.rest.issues.createComment({ | ||
issue_number: context.issue.number, | ||
owner: context.repo.owner, | ||
repo: context.repo.repo, | ||
body: `${{ steps.build-and-run.outputs.comment }}` | ||
}) | ||
- name: Comment on PR if job failed. | ||
if: failure() | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
github.rest.issues.createComment({ | ||
issue_number: context.issue.number, | ||
owner: context.repo.owner, | ||
repo: context.repo.repo, | ||
body: `There was a problem while building and running in standalone mode. You can find a log of the job [here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}).` | ||
}) | ||
|
||
cmssw: | ||
if: > | ||
github.event.issue.pull_request && | ||
contains(github.event.comment.body, '/run cmssw') && | ||
(github.event.comment.author_association == 'MEMBER' || github.event.comment.author_association == 'CONTRIBUTOR') | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout PR | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: refs/pull/${{ github.event.issue.number }}/head | ||
- name: Get CMSSW branch name | ||
env: | ||
COMMENT_BODY: ${{ github.event.comment.body }} | ||
shell: bash | ||
run: | | ||
cmssw_branch= | ||
while IFS= read -r line; do | ||
echo "$line" | ||
if [[ $line == *"/run cmssw"* ]]; then | ||
# Check if there is a branch after "/run cmssw" | ||
words=($line) | ||
cmssw_branch="${words[2]}" | ||
# Validate the extracted branch to avoid code injection | ||
if ! [[ $cmssw_branch =~ ^[[:alnum:]_-]+$ ]]; then | ||
echo "Branch name is invalid. Ignoring..." | ||
cmssw_branch= | ||
fi | ||
fi | ||
done <<< "$COMMENT_BODY" | ||
if [ -z "$cmssw_branch" ]; then | ||
cmssw_branch=default | ||
fi | ||
echo "Using CMSSW branch: $cmssw_branch" | ||
echo "cmssw-branch=$cmssw_branch" >> $GITHUB_ENV | ||
- name: Build and run PR | ||
id: build-and-run | ||
uses: SegmentLinking/TrackLooper-actions/cmssw@v1 | ||
with: | ||
pr-number: ${{ github.event.issue.number }} | ||
cmssw-branch: ${{ env.cmssw-branch }} | ||
- name: Upload plots to archival repo | ||
uses: s0/git-publish-subdir-action@develop | ||
env: | ||
REPO: [email protected]:${{ steps.build-and-run.outputs.archive-repo }}.git | ||
BRANCH: ${{ steps.build-and-run.outputs.archive-branch }} | ||
FOLDER: ${{ steps.build-and-run.outputs.archive-dir }} | ||
SSH_PRIVATE_KEY: ${{ secrets.DEPLOY_PRIVATE_KEY }} | ||
TARGET_DIR: ${{ steps.build-and-run.outputs.archive-dir }} | ||
- name: Comment on PR | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
github.rest.issues.createComment({ | ||
issue_number: context.issue.number, | ||
owner: context.repo.owner, | ||
repo: context.repo.repo, | ||
body: `${{ steps.build-and-run.outputs.comment }}` | ||
}) | ||
- name: Comment on PR if job failed. | ||
if: failure() | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
github.rest.issues.createComment({ | ||
issue_number: context.issue.number, | ||
owner: context.repo.owner, | ||
repo: context.repo.repo, | ||
body: `There was a problem while building and running with CMSSW. You can find a log of the job [here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}).` | ||
}) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there a way to run the code checks locally without the CI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you want to check formatting you can run
clang-format -i -n SDL/*.h SDL/*.cc
. To apply the changed you simply remove the-n
flag.To check with
clang-tidy
is more annoying. It should be something like this.