Migrate to GHCR (#6478)

* Replace `radius.azurecr.io` with `ghcr.io/radius-project` * Replace `radiusdev.azurecr.io` with `ghcr.io/radius-project/dev` * Add step to publish latest rad CLI to `ghcr.io` * Refactoring `Build and Test` GH workflow to improve clarity * Removing ACR publishes and pulls from GH workflows * Removing instances of azurecr.io and ACR from our docs  - This pull request adds or changes features of Radius and has an approved issue (issue link required).  Fixes: #6362 Fixes: #6295   🔄🛠️🧪  This pull request updates the GitHub workflows to use GHCR as the new registry for radius artifacts. It changes the build.yaml workflow to use new tools and commands for building and pushing the rad CLI, the radius images, and the radius helm chart to GHCR. It also changes the functional-test.yaml workflow to use GHCR for bicep recipes instead of ACR. > _We're sailing on the GitHub seas, with rad and radius tools_ > _We've changed our registry, to GHCR from ACR_ > _So heave away, me hearties, heave away with `oras` and `buildx`_ > _And don't forget to push the helm chart, with the registry plugin_ * Remove `GOPROXY` and `DOCKER_REGISTRY` environment variables and add `ORAS_VERSION` and `IMAGE_SRC` environment variables to support ORAS CLI for pushing rad CLI binaries to GHCR ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-d0777657fa3fd81d23aaf7273e58aee453b04e67882517900c56daeef9b3e4c1L39-R42)) * Rename `build` job to `build-and-push-cli` and add `build-and-push-images` job to build and push radius container images to GHCR using Docker Buildx ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-d0777657fa3fd81d23aaf7273e58aee453b04e67882517900c56daeef9b3e4c1L57-R59)) * Remove `images` job and add `publish_images` job to build and push rad CLI binaries to GHCR using ORAS CLI for different platforms and architectures ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-d0777657fa3fd81d23aaf7273e58aee453b04e67882517900c56daeef9b3e4c1L207-R226)) * Rename `helm` job to `build-and-push-helm-chart` and use Helm Registry plugin to push radius helm chart to GHCR ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-d0777657fa3fd81d23aaf7273e58aee453b04e67882517900c56daeef9b3e4c1L312-R277)) * Rename `publish_release` job to `publish-release` and change `needs` condition to depend on `build-and-push-cli` job ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-d0777657fa3fd81d23aaf7273e58aee453b04e67882517900c56daeef9b3e4c1L353-R319)) * Remove `publish` job and update `delete_artifacts` job to depend on `build-and-push-cli` job ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-d0777657fa3fd81d23aaf7273e58aee453b04e67882517900c56daeef9b3e4c1L391-R359)) * Change `BICEP_RECIPE_REGISTRY` environment variable to use GHCR instead of ACR for bicep recipes in `.github/workflows/functional-test.yaml` ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-c79f364a9293abaaa8595776b74674e24bec6287834e63ab8aa7aec6a42f0dbcL56-R56)) * Add comment to indicate that `az acr login` command needs to be changed to use GHCR in `.github/workflows/functional-test.yaml` ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-c79f364a9293abaaa8595776b74674e24bec6287834e63ab8aa7aec6a42f0dbcR81)) * Remove `az acr login` command from `.github/workflows/functional-test.yaml` as it is no longer needed ([link](https://github.com/radius-project/radius/pull/6478/files?diff=unified&w=0#diff-c79f364a9293abaaa8595776b74674e24bec6287834e63ab8aa7aec6a42f0dbcL486-L488)) --------- Co-authored-by: ytimocin <[email protected]> (cherry picked from commit a197131)
radius-project · Oct 18, 2023 · ee96908 · ee96908
1 parent 627184f
commit ee96908
Show file tree

Hide file tree

Showing 90 changed files with 318 additions and 401 deletions.
diff --git a/.github/scripts/publish-recipes.sh b/.github/scripts/publish-recipes.sh
@@ -16,32 +16,28 @@
 # limitations under the License.
 # ------------------------------------------------------------
 
-# Fail immedietly if any command fails
+# Fail immediately if any command fails
 set -e
 
 # Get command line arguments
-BICEP_PATH=$1
-DIRECTORY=$2
-REGISTRY_PATH=$3
-RECIPE_VERSION=$4
-
-BICEP_EXECUTABLE="$BICEP_PATH/rad-bicep"
+DIRECTORY=$1
+REGISTRY_PATH=$2
+RECIPE_VERSION=$3
 
 # Print usage information
 function print_usage() {
-    echo "Usage: $0 <BICEP_PATH> <DIRECTORY> <REGISTRY_PATH> <RECIPE_VERSION>"
+    echo "Usage: $0 <DIRECTORY> <REGISTRY_PATH> <RECIPE_VERSION>"
     echo ""
-    echo "  Publishes all recipes in the repository to the Azure Container Registry. Requires you to be logged into Azure via az login."
+    echo "  Publishes all recipes in the repository to a container registry."
     echo ""
-    echo "  BICEP_PATH: Path to directory containing the bicep executable. For example, ~/.rad/bin"
     echo "  DIRECTORY: Directory containing the recipes to publish. For example, ./test/functional/testdata/recipes"
-    echo "  REGISTRY_PATH: Registry hostname and path prefix. For example, myregistry.azurecr.io/tests/recipes."
+    echo "  REGISTRY_PATH: Registry hostname and path prefix. For example, ghcr.io/radius-project/dev/test/recipes"
     echo "  RECIPE_VERSION: Version of the recipe to publish. For example, pr-19293"
     echo ""
 }
 
 # Verify that the required arguments are present
-if [[ $# -ne 4 ]]; then
+if [[ $# -ne 3 ]]; then
     echo "Error: Missing required arguments"
     echo ""
     print_usage
@@ -69,5 +65,5 @@ do
 
     echo "Publishing $RECIPE to $PUBLISH_REF"
     echo "- $PUBLISH_REF" >> $GITHUB_STEP_SUMMARY
-    $BICEP_EXECUTABLE publish $RECIPE --target "br:$PUBLISH_REF"
+    rad bicep publish --file $RECIPE --target "br:$PUBLISH_REF"
 done
diff --git a/.github/scripts/release-verification.sh b/.github/scripts/release-verification.sh
@@ -59,9 +59,9 @@ fi
 kind create cluster
 ./rad install kubernetes
 
-EXPECTED_APPCORE_RP_IMAGE="radius.azurecr.io/applications-rp:${EXPECTED_TAG_VERSION}"
-EXPECTED_UCP_IMAGE="radius.azurecr.io/ucpd:${EXPECTED_TAG_VERSION}"
-EXPECTED_DE_IMAGE="radius.azurecr.io/deployment-engine:${EXPECTED_TAG_VERSION}"
+EXPECTED_APPCORE_RP_IMAGE="ghcr.io/radius-project/applications-rp:${EXPECTED_TAG_VERSION}"
+EXPECTED_UCP_IMAGE="ghcr.io/radius-project/ucpd:${EXPECTED_TAG_VERSION}"
+EXPECTED_DE_IMAGE="ghcr.io/radius-project/deployment-engine:${EXPECTED_TAG_VERSION}"
 
 APPCORE_RP_IMAGE=$(kubectl describe pods -n radius-system -l control-plane=applications-rp | awk '/^.*Image:/ {print $2}')
 UCP_IMAGE=$(kubectl describe pods -n radius-system -l control-plane=ucp | awk '/^.*Image:/ {print $2}')

diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml
@@ -36,26 +36,27 @@ concurrency:
 env:
   # Go version to install
   GOVER: '^1.21'
-  GOPROXY: https://proxy.golang.org
 
   # gotestsum version - see: https://github.com/gotestyourself/gotestsum
   GOTESTSUMVERSION: 1.10.0
 
-  # Use radiusdev.azurecr.io for PR build. Otherwise, use radius.azurecr.io.
-  # TODO_LAUNCH: Remove this variable when we opensource the repo - https://github.com/radius-project/radius/issues/5892
-  DOCKER_REGISTRY: ${{ github.event_name == 'pull_request' && 'radiusdev.azurecr.io' || 'radius.azurecr.io' }}
-
   # GitHub Actor for pushing images to GHCR
   GHCR_ACTOR: rad-ci-bot
 
   # Container registry url for GitHub container registry.
-  CONTAINER_REGISTRY: ${{ github.event_name == 'pull_request' && 'ghcr.io/radius-project/dev' || 'ghcr.io/radius-project/radius' }}
+  CONTAINER_REGISTRY: ${{ github.event_name == 'pull_request' && 'ghcr.io/radius-project/dev' || 'ghcr.io/radius-project' }}
 
   # Local file path to the release binaries.
   RELEASE_PATH: ./release
 
+  # ORAS (OCI Registry As Storage) CLI version
+  ORAS_VERSION: 1.1.0
+
+  # URL to get source code for building the image
+  IMAGE_SRC: https://github.com/radius-project/radius
+
 jobs:
-  build:
+  build-and-push-cli:
     name: Build ${{ matrix.target_os }}_${{ matrix.target_arch }} binaries
     runs-on: ubuntu-latest
     env:
@@ -126,7 +127,6 @@ jobs:
         with:
           name: rad_cli_release
           path: ${{ env.RELEASE_PATH }}
-      # TOOD_LAUNCH: Remove this step when we opensource the repo - https://github.com/radius-project/radius/issues/5892
       - name: Upload CLI binary
         uses: actions/upload-artifact@v3
         with:
@@ -204,64 +204,25 @@ jobs:
         with:
           path: ./dist/cache
           key: radius-coverage-${{ github.sha }}-${{ github.run_number }}
-
-  # Logic here:
-  # - always do a docker build for validation
-  # - tag the image as latest and with a version if the trigger was a tag
-  # - tag the image with the PR version if the trigger was a PR
-  # - push the image for pushes to master, or to a tag
-
-  # TODO_LAUNCH: Remove 'image' job when we opensource the repo - https://github.com/radius-project/radius/issues/5892
-  images:
-    name: Container image build
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check out code
-        uses: actions/checkout@v3
-      - name: Parse release version and set environment variables
-        run: python ./.github/scripts/get_release_version.py
-      - name: Set up Go ${{ env.GOVER }}
-        uses: actions/setup-go@v4
-        with:
-          go-version: ${{ env.GOVER }}
-      - uses: azure/docker-login@v1
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v2
         with:
-          login-server: ${{ env.DOCKER_REGISTRY }}
-          username: '${{ secrets.DOCKER_USERNAME }}'
-          password: '${{ secrets.DOCKER_PASSWORD }}'
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: oras-project/setup-oras@v1
         with:
-          platforms: linux/amd64,linux/arm64,linux/arm/v7
-      - name: Push container images (latest)
+          version: ${{ env.ORAS_VERSION }}
+      - name: Push latest rad cli binary to GHCR (unix-like)
+        if: github.ref == 'refs/heads/main' && matrix.target_os != 'windows'
         run: |
-          make docker-test-image-build && make docker-test-image-push
-          make docker-multi-arch-push
-        if: (github.ref == 'refs/heads/main') # push image to latest on merge to main
-        env:
-          DOCKER_REGISTRY: ${{ env.DOCKER_REGISTRY }}
-          DOCKER_TAG_VERSION: latest
-      - name: Push container images (PR)
-        run: |
-          make docker-test-image-build && make docker-test-image-push
-          make docker-multi-arch-push
-        if: github.event_name == 'pull_request'
-        env:
-          DOCKER_REGISTRY: ${{ env.DOCKER_REGISTRY }}
-          DOCKER_TAG_VERSION: ${{ env.REL_VERSION }} # includes PR number
-      - name: Push container images (release)
+          oras push ${{ env.CONTAINER_REGISTRY }}/rad/${{ matrix.target_os }}-${{ matrix.target_arch }}:latest ./dist/${{ matrix.target_os}}_${{ matrix.target_arch}}/release/rad --annotation "org.opencontainers.image.source=${{ env.IMAGE_SRC }}"
+      - name: Copy cli binaries to release (windows)
+        if: github.ref == 'refs/heads/main' && matrix.target_os == 'windows'
         run: |
-          make docker-test-image-build && make docker-test-image-push
-          make docker-multi-arch-push
-        if: startsWith(github.ref, 'refs/tags/v') # push image on tag
-        env:
-          DOCKER_REGISTRY: ${{ env.DOCKER_REGISTRY }}
-          DOCKER_TAG_VERSION: ${{ env.REL_CHANNEL }}
+          oras push ${{ env.CONTAINER_REGISTRY }}/rad/${{ matrix.target_os }}-${{ matrix.target_arch }}:latest ./dist/${{ matrix.target_os}}_${{ matrix.target_arch}}/release/rad.exe --annotation "org.opencontainers.image.source=${{ env.IMAGE_SRC }}"
 
-  # publish_image is building and publishing images to GHCR.
-  publish_images:
+  build-and-push-images:
     name: Build and publish container images
     runs-on: ubuntu-latest
     steps:
@@ -277,8 +238,8 @@ jobs:
         uses: docker/login-action@v2
         with:
           registry: ghcr.io
-          username: ${{ env.GHCR_ACTOR }}
-          password: ${{ secrets.GH_RAD_CI_BOT_PAT }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
       - name: Set up Docker Buildx
@@ -309,9 +270,10 @@ jobs:
         env:
           DOCKER_REGISTRY: ${{ env.CONTAINER_REGISTRY }}
           DOCKER_TAG_VERSION: ${{ env.REL_CHANNEL }}
-  helm:
+
+  build-and-push-helm-chart:
     name: Helm chart build
-    needs: ['images', 'publish_images']
+    needs: ['build-and-push-images']
     runs-on: ubuntu-latest
     env:
       ARTIFACT_DIR: ./dist/Charts
@@ -343,16 +305,14 @@ jobs:
             --username ${{ secrets.AZURE_SP_TESTS_APPID }} \
             --password ${{ secrets.AZURE_SP_TESTS_PASSWORD }} \
             --tenant ${{ secrets.AZURE_SP_TESTS_TENANTID }}
-      - name: Push helm chart to ACR
-        run: |
-          az acr helm push --name radius ${{ env.ARTIFACT_DIR }}/${{ env.HELM_PACKAGE_DIR }}/radius-${{ env.CHART_VERSION }}.tgz --force
       - name: Push helm chart to GHCR
         run: |
           echo ${{ secrets.GITHUB_TOKEN }} | helm registry login -u ${{ github.actor }} --password-stdin ${{ env.OCI_REGISTRY }}
           helm push ${{ env.ARTIFACT_DIR }}/${{ env.HELM_PACKAGE_DIR }}/radius-${{ env.CHART_VERSION }}.tgz oci://${{ env.OCI_REGISTRY }}/${{ env.OCI_REPOSITORY }}
-  publish_release:
+
+  publish-release:
     name: Publish GitHub Release
-    needs: [ 'build' ]
+    needs: ['build-and-push-cli']
     runs-on: ubuntu-latest
     if: startsWith(github.ref, 'refs/tags/v')
     steps:
@@ -376,8 +336,6 @@ jobs:
             --generate-notes \
             --verify-tag \
             --prerelease
-        env:
-          GITHUB_TOKEN: ${{ secrets.GH_RAD_CI_BOT_PAT }}
       - name: Create GitHub Official Release
         if: ${{ !contains(env.REL_VERSION, 'rc') }}
         run: |
@@ -386,96 +344,11 @@ jobs:
             --title "Project Radius v${{ env.REL_VERSION }}" \
             --notes-file docs/release-notes/v${{ env.REL_VERSION }}.md \
             --verify-tag
-        env:
-          GITHUB_TOKEN: ${{ secrets.GH_RAD_CI_BOT_PAT }}
-  # TODO_LAUNCH: Remove this job once we opensource - https://github.com/radius-project/radius/issues/5892
-  publish:
-    name: Publish rad CLI binaries
-    needs: [ 'build' ]
-    runs-on: ubuntu-latest
-    if: (github.ref == 'refs/heads/main') || startsWith(github.ref, 'refs/tags/v') # upload on push to main or tag
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-      - name: Parse release version and set environment variables
-        run: python ./.github/scripts/get_release_version.py
-      - name: Download release artifacts
-        uses: actions/download-artifact@v3
-        with:
-          name: rad_cli_darwin_arm64
-          path: rad_cli_darwin_arm64
-      - name: Download release artifacts
-        uses: actions/download-artifact@v3
-        with:
-          name: rad_cli_darwin_amd64
-          path: rad_cli_darwin_amd64
-      - name: Download release artifacts
-        uses: actions/download-artifact@v3
-        with:
-          name: rad_cli_linux_arm
-          path: rad_cli_linux_arm
-      - name: Download release artifacts
-        uses: actions/download-artifact@v3
-        with:
-          name: rad_cli_linux_arm64
-          path: rad_cli_linux_arm64
-      - name: Download release artifacts
-        uses: actions/download-artifact@v3
-        with:
-          name: rad_cli_linux_amd64
-          path: rad_cli_linux_amd64
-      - name: Download release artifacts
-        uses: actions/download-artifact@v3
-        with:
-          name: rad_cli_windows_amd64
-          path: rad_cli_windows_amd64
-      - uses: bacongobbler/[email protected]
-        with:
-          source_dir: rad_cli_darwin_amd64
-          container_name: 'tools'
-          connection_string: ${{ secrets.ASSETS_STORAGE_CONNECTION_STRING }}
-          overwrite: 'true'
-          extra_args: '--destination-path rad/${{ env.REL_CHANNEL }}/macos-x64/ --pattern rad --timeout 300'
-      - uses: bacongobbler/[email protected]
-        with:
-          source_dir: rad_cli_darwin_arm64
-          container_name: 'tools'
-          connection_string: ${{ secrets.ASSETS_STORAGE_CONNECTION_STRING }}
-          overwrite: 'true'
-          extra_args: '--destination-path rad/${{ env.REL_CHANNEL }}/macos-arm64/ --pattern rad --timeout 300'
-      - uses: bacongobbler/[email protected]
-        with:
-          source_dir: rad_cli_linux_amd64
-          container_name: 'tools'
-          connection_string: ${{ secrets.ASSETS_STORAGE_CONNECTION_STRING }}
-          overwrite: 'true'
-          extra_args: '--destination-path rad/${{ env.REL_CHANNEL }}/linux-x64/ --pattern rad --timeout 300'
-      - uses: bacongobbler/[email protected]
-        with:
-          source_dir: rad_cli_linux_arm
-          container_name: 'tools'
-          connection_string: ${{ secrets.ASSETS_STORAGE_CONNECTION_STRING }}
-          overwrite: 'true'
-          extra_args: '--destination-path rad/${{ env.REL_CHANNEL }}/linux-arm/ --pattern rad --timeout 300'
-      - uses: bacongobbler/[email protected]
-        with:
-          source_dir: rad_cli_linux_arm64
-          container_name: 'tools'
-          connection_string: ${{ secrets.ASSETS_STORAGE_CONNECTION_STRING }}
-          overwrite: 'true'
-          extra_args: '--destination-path rad/${{ env.REL_CHANNEL }}/linux-arm64/ --pattern rad --timeout 300'
-      - uses: bacongobbler/[email protected]
-        with:
-          source_dir: rad_cli_windows_amd64
-          container_name: 'tools'
-          connection_string: ${{ secrets.ASSETS_STORAGE_CONNECTION_STRING }}
-          overwrite: 'true'
-          extra_args: '--destination-path rad/${{ env.REL_CHANNEL }}/windows-x64/ --pattern rad.exe --timeout 300'
 
   delete_artifacts:
     name: Delete artifacts
-    needs: [ 'build', 'publish' ]
-    if: ${{ always() && !contains(needs.build.result, 'failure') }}
+    needs: ['build-and-push-cli']
+    if: ${{ always() && !contains(needs.build-and-push-cli.result, 'failure') }}
     runs-on: ubuntu-latest
     steps:
       - name: Delete release artifacts