Remove e2e tests not required label #11183
Conversation
|
It used to be:
Now it's:
|
There was a problem hiding this comment.
I love the documentation update it's really good.
I ran it in this branch here https://buildkite.com/wellcomecollection/wc-dot-org-build-plus-test/builds/14794#_ and it works nicely 🙌
Only question is that the "Add steps for e2e tests (if necessary)" step didn't use to wait until all the steps above were finished (I'm pretty sure anyway but might be wrong) and now it does, which might slow down things a minute or two. Which is pretty minimal and might've been a one-off, so very much not a blocker.
| if [[ "$BUILDKITE_PULL_REQUEST" != "false" ]] | ||
| then | ||
| buildkite-agent pipeline upload << EOF | ||
| - wait |
There was a problem hiding this comment.
@rcantin-w I carried over this wait (and the one below on line 75) into the main pipeline.yml so the behaviour should be the same as before. I think we actually have to wait until everything is built before deploying into the e2e env
What does this change?
Part of wellcomecollection/platform#5740
We're setting up alternatives to weco-bot everywhere it's used because it doesn't conform the the SSO standard we're set. Weco-bot is a "fake" team member that has its own credentials, used to perform automated tasks.
Before this change, one of these tasks was to call Github API from a Buildkite pipeline run, in order to add the "e2e not required" label to the PR. This label was only used to "remember" on later commits if the developer had asked that e2e tests never be run on the PR.
In the absence of this "e2e not required" label, Buildkite will ask if the dev wants to run the e2e tests or not every time a commit is built.
As agreed with the experience team, we're removing the option to add the "e2e tests not required" label on PRs from Buildkite.
After this change, every time a commit is pushed to a branch and triggers a build & test in Buildkite, the dev will need to select whether or not they want e2e tests run in order to complete the pipeline.
How to test
Checkout a new branch, make any trivial change and push to the remote. In https://buildkite.com/wellcomecollection/wc-dot-org-build-plus-test you should see you commit building, with a step asking you whether you want e2e tests or not.
Select Yes and see that the tests run and the pipeline completes.
Push another commit to the branch, select No and see that the pipeline completes without running the e2e tests
How can we measure success?
Weco-bot is not involved in running/managing e2e tests on PR branches
Have we considered potential risks?
The dev still has to make the deliberate decision to NOT run the e2e tests for the pipeline to complete