Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Randomize the id with GITHUB_RUN_NUMBER for scheduled test #7392

Merged
merged 7 commits into from
Mar 25, 2024

Conversation

youngbupark
Copy link

@youngbupark youngbupark commented Mar 25, 2024

Description

We generate unique id by concatenating GitHub Actions' system variables, hashing the string, and taking 10 characters. This way will reduce the chance to create multiple resource for the same PR. In scheduled test, the most of variables are same, so there is a little chance to face the conflicts. particularly, when resource group is being in progress triggered by the previous action runs like below failures. This change 1) adds GITHUB_RUN_NUMBER, which is updated for each action run, to the raw id before hashing and 2) adds the prefix to each hashed string to avoid the conflicts between different workflows.

#7383
#7382
#7380

Type of change

  • This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: #7388

@radius-functional-tests
Copy link

radius-functional-tests bot commented Mar 25, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref d1f523a
Unique ID b192a768de
Image tag pr-b192a768de
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-b192a768de
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-b192a768de
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-b192a768de
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-b192a768de
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting cli functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting ucp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting samples functional tests...
✅ kubernetes functional tests succeeded
✅ msgrp functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ shared functional tests succeeded

Signed-off-by: Young Bu Park <[email protected]>
Copy link

github-actions bot commented Mar 25, 2024

Test Results

3 135 tests  ±0   3 128 ✅ ±0   3m 30s ⏱️ -14s
  261 suites ±0       7 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit b00b047. ± Comparison against base commit 28f73b7.

♻️ This comment has been updated with latest results.

Copy link

65.9

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 65.9 %
  • main branch coverage: 65.9 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

Signed-off-by: Young Bu Park <[email protected]>
Signed-off-by: Young Bu Park <[email protected]>
Copy link

65.9

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 65.9 %
  • main branch coverage: 65.9 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

Signed-off-by: Young Bu Park <[email protected]>
Signed-off-by: Young Bu Park <[email protected]>
@youngbupark youngbupark marked this pull request as ready for review March 25, 2024 21:13
@youngbupark youngbupark requested review from a team as code owners March 25, 2024 21:13
@radius-functional-tests
Copy link

radius-functional-tests bot commented Mar 25, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref b00b047
Unique ID d4a679b153
Image tag pr-d4a679b153
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-d4a679b153
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-d4a679b153
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-d4a679b153
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-d4a679b153
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting ucp functional tests...
⌛ Starting samples functional tests...
⌛ Starting shared functional tests...
✅ msgrp functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ cli functional tests succeeded
✅ shared functional tests succeeded

UNIQUE_ID=$(echo $BASE_STR | sha1sum | head -c 10)
if [ "$GITHUB_EVENT_NAME" == "schedule" ]; then
# Add run number to randomize unique id for scheduled runs.
BASE_STR="${GITHUB_RUN_NUMBER}|${BASE_STR}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any reason we only do this for scheduled runs? is there a downside to adding the run number on all runs?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use GITHUB_RUN_NUMBER for PR event, workflow will create resource groups and resources whenever we trigger functional tests. This results in unnecessary resource creation in our subscription. To avoid this issue and save the cost, we do not use GITHUB_RUN_NUMBER for every event.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though we have purge workflow, it cleans up only 3 days+ old resources. So we still need to pay 3 days resoruce usage. so it is better not to use GITHUB_RUN_NUMBER for every event.

Copy link

65.9

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 65.9 %
  • main branch coverage: 65.9 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@youngbupark youngbupark merged commit 166d872 into main Mar 25, 2024
20 of 21 checks passed
@youngbupark youngbupark deleted the youngp/random-name branch March 25, 2024 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Randomize test resource name for scheduled functional test
3 participants