From 31487d3274b6cebf37cf96fac0c0189768ac34ad Mon Sep 17 00:00:00 2001 From: mkolasinski-splunk Date: Thu, 27 Jun 2024 09:28:24 +0200 Subject: [PATCH 1/3] chore: extend docker image documentation by unreleased splunk --- ...ry-workflow-addon-release-docker-images.md | 26 +++++++++++++++++++ ...lish_multiple_os_images_scripted_inputs.md | 12 --------- 2 files changed, 26 insertions(+), 12 deletions(-) create mode 100644 runbooks/addonfactory-workflow-addon-release-docker-images.md delete mode 100644 runbooks/publish_multiple_os_images_scripted_inputs.md diff --git a/runbooks/addonfactory-workflow-addon-release-docker-images.md b/runbooks/addonfactory-workflow-addon-release-docker-images.md new file mode 100644 index 000000000..034a30699 --- /dev/null +++ b/runbooks/addonfactory-workflow-addon-release-docker-images.md @@ -0,0 +1,26 @@ +# Runbook to creating and publishing docker images used in reusable workflow +## Runbook to publish multiple images of different Linux flavors and versions for scripted inputs tests +Once there is new Splunk release, and [matrix](https://github.com/splunk/addonfactory-test-matrix-action) is updated, we need to make sure that Splunk images for scripted inputs tests are created and published. +### Steps + +#### Update OS images +- check what OS are listed in definition of matrix in scripted inputs tests [here](https://github.com/splunk/addonfactory-workflow-addon-release/blob/v4.16/.github/workflows/reusable-build-test-release.yml#L1966) +- if any is missing in [ta-automation-docker-images](https://cd.splunkdev.com/taautomation/ta-automation-docker-images/-/tree/main/dockerfiles) then add new Dockerfile + +#### Create images and publish them to ECR +- figure out what version of Splunk is needed (sha) using go/fetcher +- trigger [pipeline](https://cd.splunkdev.com/taautomation/ta-automation-docker-images/-/pipelines/new) for every OS flavor separately + +## Runbook to publish unreleased splunk image for testing +Whenever there is a need for running tests with unreleased splunk, we need to create relevant splunk docker image and publish it to aws ecr +### Steps +#### Build docker image and publish to artifactory +- Prior creating docker image it needs to be determined which revision of core splunk repo is required. Splunk docker images are based on splunk builds published to artifactory by CI in core repository. Their names match SHA of the commit in core repo: [develop builds artifactory](https://repo.splunkdev.net:443/artifactory/generic/splcore/builds/develop/) +- Docker image is built by [pipeline](https://cd.splunkdev.com/core-ee/docker-splunk-internal/-/pipelines/new) which required UNRELEASED_SPLUNK_SHA as an input variable - provide first 12 characters of desired revision on splunk core repo. Once image is built, it is published to [artifactory](https://repo.splunkdev.net/ui/repos/tree/General/docker/docker-splunk-internal/unreleased/splunk-redhat-9). +#### Pull built image locally, tag and publish to ecr +- docker pull docker.repo.splunkdev.net/docker-splunk-internal/unreleased/splunk-redhat-9:[image-tag] +- docker tag docker.repo.splunkdev.net/docker-splunk-internal/unreleased/splunk-redhat-9:[image-tag] "956110764581.dkr.ecr.us-west-2.amazonaws.com/splunk/splunk:[new-image-tag]" +- set AWS environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN +- aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 956110764581.dkr.ecr.us-west-2.amazonaws.com +- docker push 956110764581.dkr.ecr.us-west-2.amazonaws.com/splunk/splunk:[new-image-tag] +- confirm that image is visible in AWS [ECR](https://us-west-2.console.aws.amazon.com/ecr/repositories/private/956110764581/splunk/splunk?region=us-west-2) \ No newline at end of file diff --git a/runbooks/publish_multiple_os_images_scripted_inputs.md b/runbooks/publish_multiple_os_images_scripted_inputs.md deleted file mode 100644 index 729ab7436..000000000 --- a/runbooks/publish_multiple_os_images_scripted_inputs.md +++ /dev/null @@ -1,12 +0,0 @@ -# Runbook to publish multiple images of different Linux flavors and versions for scripted inputs tests - -Once there is new Splunk release, and [matrix](https://github.com/splunk/addonfactory-test-matrix-action) is updated, we need to make sure that Splunk images for scripted inputs tests are created and published. -## Steps - -### Update OS images -- check what OS are listed in definition of matrix in scripted inputs tests [here](https://github.com/splunk/addonfactory-workflow-addon-release/blob/v4.16/.github/workflows/reusable-build-test-release.yml#L1966) -- if any is missing in [ta-automation-docker-images](https://cd.splunkdev.com/taautomation/ta-automation-docker-images/-/tree/main/dockerfiles) then add new Dockerfile - -### Create images and publish them to ECR -- figure out what version of Splunk is needed (sha) using go/fetcher -- trigger [pipeline](https://cd.splunkdev.com/taautomation/ta-automation-docker-images/-/pipelines/new) for every OS flavor separately From 256f9a5440686400f83515fbba38d182e7ec55de Mon Sep 17 00:00:00 2001 From: mkolasinski-splunk Date: Fri, 12 Jul 2024 10:29:27 +0200 Subject: [PATCH 2/3] chore: update link to unix test oss --- runbooks/addonfactory-workflow-addon-release-docker-images.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/runbooks/addonfactory-workflow-addon-release-docker-images.md b/runbooks/addonfactory-workflow-addon-release-docker-images.md index 034a30699..48e169fc3 100644 --- a/runbooks/addonfactory-workflow-addon-release-docker-images.md +++ b/runbooks/addonfactory-workflow-addon-release-docker-images.md @@ -4,7 +4,7 @@ Once there is new Splunk release, and [matrix](https://github.com/splunk/addonfa ### Steps #### Update OS images -- check what OS are listed in definition of matrix in scripted inputs tests [here](https://github.com/splunk/addonfactory-workflow-addon-release/blob/v4.16/.github/workflows/reusable-build-test-release.yml#L1966) +- check what OSs are listed in definition of matrix for scripted inputs tests [here](https://github.com/splunk/addonfactory-workflow-addon-release/blob/72497e5c03894369b8fbdd2a2c4134c233ba1b5d/.github/workflows/reusable-build-test-release.yml#L27) - if any is missing in [ta-automation-docker-images](https://cd.splunkdev.com/taautomation/ta-automation-docker-images/-/tree/main/dockerfiles) then add new Dockerfile #### Create images and publish them to ECR @@ -18,6 +18,7 @@ Whenever there is a need for running tests with unreleased splunk, we need to cr - Prior creating docker image it needs to be determined which revision of core splunk repo is required. Splunk docker images are based on splunk builds published to artifactory by CI in core repository. Their names match SHA of the commit in core repo: [develop builds artifactory](https://repo.splunkdev.net:443/artifactory/generic/splcore/builds/develop/) - Docker image is built by [pipeline](https://cd.splunkdev.com/core-ee/docker-splunk-internal/-/pipelines/new) which required UNRELEASED_SPLUNK_SHA as an input variable - provide first 12 characters of desired revision on splunk core repo. Once image is built, it is published to [artifactory](https://repo.splunkdev.net/ui/repos/tree/General/docker/docker-splunk-internal/unreleased/splunk-redhat-9). #### Pull built image locally, tag and publish to ecr +- okta-artifactory-login -t docker - docker pull docker.repo.splunkdev.net/docker-splunk-internal/unreleased/splunk-redhat-9:[image-tag] - docker tag docker.repo.splunkdev.net/docker-splunk-internal/unreleased/splunk-redhat-9:[image-tag] "956110764581.dkr.ecr.us-west-2.amazonaws.com/splunk/splunk:[new-image-tag]" - set AWS environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN From dd1ce195931034fc90fc23040f75430555f21531 Mon Sep 17 00:00:00 2001 From: mkolasinski-splunk Date: Fri, 12 Jul 2024 11:09:03 +0200 Subject: [PATCH 3/3] chore: respect the Splunk --- .../addonfactory-workflow-addon-release-docker-images.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/runbooks/addonfactory-workflow-addon-release-docker-images.md b/runbooks/addonfactory-workflow-addon-release-docker-images.md index 48e169fc3..66eae32cc 100644 --- a/runbooks/addonfactory-workflow-addon-release-docker-images.md +++ b/runbooks/addonfactory-workflow-addon-release-docker-images.md @@ -11,12 +11,12 @@ Once there is new Splunk release, and [matrix](https://github.com/splunk/addonfa - figure out what version of Splunk is needed (sha) using go/fetcher - trigger [pipeline](https://cd.splunkdev.com/taautomation/ta-automation-docker-images/-/pipelines/new) for every OS flavor separately -## Runbook to publish unreleased splunk image for testing -Whenever there is a need for running tests with unreleased splunk, we need to create relevant splunk docker image and publish it to aws ecr +## Runbook to publish unreleased Splunk image for testing +Whenever there is a need for running tests with unreleased splunk, we need to create relevant Splunk docker image and publish it to aws ecr ### Steps #### Build docker image and publish to artifactory -- Prior creating docker image it needs to be determined which revision of core splunk repo is required. Splunk docker images are based on splunk builds published to artifactory by CI in core repository. Their names match SHA of the commit in core repo: [develop builds artifactory](https://repo.splunkdev.net:443/artifactory/generic/splcore/builds/develop/) -- Docker image is built by [pipeline](https://cd.splunkdev.com/core-ee/docker-splunk-internal/-/pipelines/new) which required UNRELEASED_SPLUNK_SHA as an input variable - provide first 12 characters of desired revision on splunk core repo. Once image is built, it is published to [artifactory](https://repo.splunkdev.net/ui/repos/tree/General/docker/docker-splunk-internal/unreleased/splunk-redhat-9). +- Prior creating docker image it needs to be determined which revision of core Splunk repo is required. Splunk docker images are based on Splunk builds published to artifactory by CI in core repository. Their names match SHA of the commit in core repo: [develop builds artifactory](https://repo.splunkdev.net:443/artifactory/generic/splcore/builds/develop/) +- Docker image is built by [pipeline](https://cd.splunkdev.com/core-ee/docker-splunk-internal/-/pipelines/new) which required UNRELEASED_SPLUNK_SHA as an input variable - provide first 12 characters of desired revision on Splunk core repo. Once image is built, it is published to [artifactory](https://repo.splunkdev.net/ui/repos/tree/General/docker/docker-splunk-internal/unreleased/splunk-redhat-9). #### Pull built image locally, tag and publish to ecr - okta-artifactory-login -t docker - docker pull docker.repo.splunkdev.net/docker-splunk-internal/unreleased/splunk-redhat-9:[image-tag]