-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flaky test: Test_Render_Multiple_Routes #6263
Comments
This happened again https://github.com/radius-project/radius/actions/runs/6203745054/job/16844827955?pr=6272. I'm going to label this for Nov release since I think this flakiness is causing ops pain and should be prioritized. |
Assigning back to Will since he has a fix by ordering resources in #6262 (comment) |
willdavsmith
added a commit
that referenced
this issue
Sep 26, 2023
# Description Today we blindly delete all AWS resources created by our tests at midnight every day. This could cause functional tests to fail around this time. This PR adds a `creationTimestamp` to each AWS test resource that we create as well as logic to delete them by timestamp in the GitHub workflow. * Adds AWS creation timestamp to AWS test resources * Update purge-aws-test-resources GitHub workflow to delete old resources instead of all resources ## Type of change <!-- Please select **one** of the following options that describes your change and delete the others. Clearly identifying the type of change you are making will help us review your PR faster, and is used in authoring release notes. If you are making a bug fix or functionality change to Radius and do not have an associated issue link please create one now. --> - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional). Fixes: #6263 <!-- Please update the following to link the associated issue. This is required for some kinds of changes (see above). --> ## Auto-generated summary <!-- GitHub Copilot for docs will auto-generate a summary of the PR --> <!-- copilot:all --> ### <samp>🤖 Generated by Copilot at adea799</samp> ### Summary 🧹🕒🛠️ <!-- 1. 🧹 - This emoji represents the cleanup of old AWS resources and the improvement of test reliability. 2. 🕒 - This emoji represents the use of creation timestamps to tag and delete AWS resources based on their age. 3. 🛠️ - This emoji represents the refactoring of the workflow and the test code to use a common script and a functional package. --> This pull request improves the cleanup of AWS resources created by functional tests. It adds a `creationTimestamp` parameter to the bicep templates and tags the resources with their creation time. It also refactors the workflow to use a common script that deletes old resources based on their tags. It renames a test file and adds some helper functions to the functional package. > _We're sailing on the AWS sea, testing our bicep templates_ > _We tag our resources with `creationTimestamp`, so we don't forget_ > _When the tests are done, we run a script to clean up the mess_ > _We heave away, haul away, delete them all on the count of three_ ### Walkthrough * Add a script to delete old AWS resources created by tests ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-5625fc3520f54a6e805114f672e280334dac76957f1f5d54cbf3b661f9efeb12R1-R99)) * Update the workflow to use the script instead of inline bash code ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-cdbef2239fc7a1aa34d773cb6dae3395193da5015c322ef49bb02025b701a0bcL26-R30), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-cdbef2239fc7a1aa34d773cb6dae3395193da5015c322ef49bb02025b701a0bcL38-R41)) * Add a creationTimestamp parameter to the bicep templates and the deploy executors for the AWS tests ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-531a6f46ae9d530508fc569f3d8efb677f5c2b85ef21732415c71fd370f62050L32-R37), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-531a6f46ae9d530508fc569f3d8efb677f5c2b85ef21732415c71fd370f62050L59-R61), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-531a6f46ae9d530508fc569f3d8efb677f5c2b85ef21732415c71fd370f62050L89-R95), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-531a6f46ae9d530508fc569f3d8efb677f5c2b85ef21732415c71fd370f62050L112-R115), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-b3530585153a4510be532a01249257b09a772f91060a2d1a8be8b3d83fbb8e87L3-R6), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-b3530585153a4510be532a01249257b09a772f91060a2d1a8be8b3d83fbb8e87L9-R20), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-b3530585153a4510be532a01249257b09a772f91060a2d1a8be8b3d83fbb8e87R41-R46), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-e77e6e04f4821e32634940d0dc5c5b2779a7cff9c5c6a0bd27186c64cafbca88L3-R6), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-e77e6e04f4821e32634940d0dc5c5b2779a7cff9c5c6a0bd27186c64cafbca88L9-R20), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-e77e6e04f4821e32634940d0dc5c5b2779a7cff9c5c6a0bd27186c64cafbca88R41-R46), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-ba3f86a7fb8241bd0809f2962fd7561f641229cd4ed710bd00cba541ab66869dR3), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-ba3f86a7fb8241bd0809f2962fd7561f641229cd4ed710bd00cba541ab66869dR15-R18), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-863bee2e7a81fc1dedef482aec1f9d5f965b5a99017411c312d5a6ecf4a34e2cR3), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-863bee2e7a81fc1dedef482aec1f9d5f965b5a99017411c312d5a6ecf4a34e2cR15-R18), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-a6fbd74775c3d83d1c5b066748f75618dea663e00f90a79c8dd115bf452bcbe0L33-R39), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-fbc42064371b122a7ca2d92bfbd5452a51f680c7face11e8db22a71cd7d58e5bL30-R35), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-fbc42064371b122a7ca2d92bfbd5452a51f680c7face11e8db22a71cd7d58e5bL64-R70), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-ea6b32716a499d0deffb7fc13fe99ab1fb5b4fa7f67576e4b89757e76d813b5bL112-R114), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-ea6b32716a499d0deffb7fc13fe99ab1fb5b4fa7f67576e4b89757e76d813b5bL119-R121), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-78e80694bb1ed5b7778407307535be7c50101bfe8b07ee9961e874f4b722c582R3), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-78e80694bb1ed5b7778407307535be7c50101bfe8b07ee9961e874f4b722c582L24-R32), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-43e2af67f3b9b4ceac2bc9597de95219b945ff2a646c5f82dab83a42a3910f6cR3), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-43e2af67f3b9b4ceac2bc9597de95219b945ff2a646c5f82dab83a42a3910f6cR15-R18), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-8028150b2b6a967b9e999213bca3ca236d9428ae62acf3ba5ca2d5d1937fea80R3), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-8028150b2b6a967b9e999213bca3ca236d9428ae62acf3ba5ca2d5d1937fea80R10-R15)) * Add a Tags property to the AWS resources in the bicep templates with the key 'RadiusCreationTimestamp' and the value of the creationTimestamp parameter ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-b3530585153a4510be532a01249257b09a772f91060a2d1a8be8b3d83fbb8e87L9-R20), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-b3530585153a4510be532a01249257b09a772f91060a2d1a8be8b3d83fbb8e87R41-R46), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-e77e6e04f4821e32634940d0dc5c5b2779a7cff9c5c6a0bd27186c64cafbca88L9-R20), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-e77e6e04f4821e32634940d0dc5c5b2779a7cff9c5c6a0bd27186c64cafbca88R41-R46), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-ba3f86a7fb8241bd0809f2962fd7561f641229cd4ed710bd00cba541ab66869dR15-R18), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-863bee2e7a81fc1dedef482aec1f9d5f965b5a99017411c312d5a6ecf4a34e2cR15-R18), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-78e80694bb1ed5b7778407307535be7c50101bfe8b07ee9961e874f4b722c582L24-R32), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-43e2af67f3b9b4ceac2bc9597de95219b945ff2a646c5f82dab83a42a3910f6cR15-R18), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-8028150b2b6a967b9e999213bca3ca236d9428ae62acf3ba5ca2d5d1937fea80R10-R15)) * Add utility functions to the functional package to get the creation timestamp and generate a unique S3 bucket name ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-d7fffa085c17b99a173ae530015cc075c39419e8ff504aea56ef80eb527b3c18R321-R330)) * Replace the generateS3BucketName function with the one from the functional package ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-fbc42064371b122a7ca2d92bfbd5452a51f680c7face11e8db22a71cd7d58e5bL120-L123)) * Add import statements to use the functional and time packages where needed ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-531a6f46ae9d530508fc569f3d8efb677f5c2b85ef21732415c71fd370f62050R24), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-a6fbd74775c3d83d1c5b066748f75618dea663e00f90a79c8dd115bf452bcbe0L20-R22), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-fbc42064371b122a7ca2d92bfbd5452a51f680c7face11e8db22a71cd7d58e5bL22-R22), [link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-d7fffa085c17b99a173ae530015cc075c39419e8ff504aea56ef80eb527b3c18R29)) * Rename the file `test/functional/shared/resources/kubmetadata_cascade_test.go` to `test/functional/shared/resources/kubemetadata_cascade_test.go` ([link](https://github.com/radius-project/radius/pull/6262/files?diff=unified&w=0#diff-1e27d371cd4d9e9f238ec3ce2d2d567b1fa40156dbe16b83f2f09541c4ecce09))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
AB#9337
The text was updated successfully, but these errors were encountered: