Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: move SDM image publish to dedicated step #83

Merged
merged 7 commits into from
Nov 22, 2024

Conversation

aaronsteers
Copy link
Contributor

@aaronsteers aaronsteers commented Nov 22, 2024

The blast radius for SDM deploys is too large for us to constantly deploy. This moves the process to a single-step process that we can trigger.

How publishing would work with the workflow trigger:

  1. If 'dry_run' is checked, all steps will be evaluated, except that nothing will be published to DockerHub.
  2. You would need to run from a published version tag or define your own version tag.
  3. If the version tag input is omitted and we aren't running from a release tag, then we'll just publish a SHA-tagged image. (Still useful for testing.)
  4. If the version being release is not a prerelease version (doesn't have 'rc' or 'beta' or 'dev' in the image name), then we'll also tag as "latest".

TODO:

  • We have to basically merge this PR in order to be able to test it...
  • Consider also kicking off the version bump PR to the Builder (airbyte-platform-internal repo) when this flow with run on a non-prelease version.
  • Make sure that if the workflow is run with an explicit version tag that we also tell Poetry dynamic versioning about it so the built package has the correct version attached. (Not necessary when running from a release tag.)
  • Consider removing the version ref input entirely and just requiring a prerelease tag to be created in the repo for pre-relases.

Normal Releases

The preferred way to run this (for production releases) would be to select an already-deployed release tag as in this screenshot:

image

In that case, you only have to select the release tag and run the workflow from there.

To add:

  • In the same flow, we can auto-create the downstream PR for the Builder to use this version of the CDK.

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Introduced a new GitHub Actions workflow for publishing the Source-Declarative-Manifest (SDM) connector as a Docker image.
  • Bug Fixes

    • Enhanced error handling and version management in the Docker publishing workflow.
  • Chores

    • Removed the DockerHub publishing job from the PyPI publishing workflow, streamlining the process to focus solely on Python package publishing.

@github-actions github-actions bot added the ci label Nov 22, 2024
@aaronsteers aaronsteers marked this pull request as ready for review November 22, 2024 18:15
Copy link
Contributor

coderabbitai bot commented Nov 22, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

This pull request introduces a new GitHub Actions workflow file, publish_sdm_connector.yml, designed to automate the publishing of the Source-Declarative-Manifest (SDM) connector as a Docker image to DockerHub. It features two main jobs: build, which prepares the Docker image, and publish_sdm, which manages the Docker image publishing process. Additionally, the existing workflow for publishing to DockerHub in pypi_publish.yml has been modified to remove the publish_sdm job, streamlining the focus on Python package publishing.

Changes

File Change Summary
.github/workflows/publish_sdm_connector.yml Added a new workflow for publishing the SDM connector as a Docker image with version management.
.github/workflows/pypi_publish.yml Removed the publish_sdm job related to DockerHub publishing, retaining only Python package publishing.

Possibly related issues

Possibly related PRs

Suggested labels

enhancement

Suggested reviewers

  • ChristoGrab

Warning

Rate limit exceeded

@aaronsteers has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 24 minutes and 54 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 832c57c and 975b396.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (3)
.github/workflows/publish_sdm_connector.yml (3)

24-39: Consider adding pip caching for better performance.

The build job looks good! Would you consider adding pip caching to speed up the builds? wdyt?

  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

+     - uses: actions/setup-python@v4
+       with:
+         python-version: '3.x'
+
+     - uses: actions/cache@v3
+       with:
+         path: ~/.cache/pip
+         key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}
+         restore-keys: |
+           ${{ runner.os }}-pip-

      - uses: hynek/build-and-inspect-python-package@v2

98-164: Consider enabling Docker layer caching.

The Docker build steps look good! To speed up builds, would you consider enabling Docker layer caching? wdyt?

  - name: Build and push (version tag)
    if: env.VERSION != '' && github.event.inputs.dry_run == 'false'
    uses: docker/build-push-action@v5
    with:
      context: .
      platforms: linux/amd64,linux/arm64
      push: true
+     cache-from: type=gha
+     cache-to: type=gha,mode=max
      tags: |
        airbyte/source-declarative-manifest:${{ env.VERSION }}

1-164: Consider adding container vulnerability scanning.

Since this is a dedicated workflow for publishing Docker images, would you consider adding a vulnerability scanning step using tools like Trivy or Snyk? This would help catch any security issues before publishing. wdyt?

Example implementation:

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'airbyte/source-declarative-manifest:${{ env.VERSION }}'
          format: 'table'
          exit-code: '1'
          ignore-unfixed: true
          vuln-type: 'os,library'
          severity: 'CRITICAL,HIGH'
🧰 Tools
🪛 actionlint (1.7.3)

56-56: shellcheck reported issue in this script: SC2086:info:6:39: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC1073:error:12:1: Couldn't parse this if expression. Fix to allow more checks

(shellcheck)


66-66: shellcheck reported issue in this script: SC1050:error:12:31: Expected 'then'

(shellcheck)


66-66: shellcheck reported issue in this script: SC1072:error:12:31: Expected 'then'. Fix any mentioned problems and try again

(shellcheck)


66-66: shellcheck reported issue in this script: SC1140:error:12:31: Unexpected parameters after condition. Missing &&/||, or bad expression?

(shellcheck)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 2368b59 and 953e66d.

📒 Files selected for processing (2)
  • .github/workflows/publish_sdm_connector.yml (1 hunks)
  • .github/workflows/pypi_publish.yml (0 hunks)
💤 Files with no reviewable changes (1)
  • .github/workflows/pypi_publish.yml
🧰 Additional context used
🪛 actionlint (1.7.3)
.github/workflows/publish_sdm_connector.yml

56-56: shellcheck reported issue in this script: SC2086:info:6:39: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC1073:error:12:1: Couldn't parse this if expression. Fix to allow more checks

(shellcheck)


66-66: shellcheck reported issue in this script: SC1050:error:12:31: Expected 'then'

(shellcheck)


66-66: shellcheck reported issue in this script: SC1072:error:12:31: Expected 'then'. Fix any mentioned problems and try again

(shellcheck)


66-66: shellcheck reported issue in this script: SC1140:error:12:31: Unexpected parameters after condition. Missing &&/||, or bad expression?

(shellcheck)

🔇 Additional comments (2)
.github/workflows/publish_sdm_connector.yml (2)

1-22: LGTM! Well-structured workflow configuration.

The workflow metadata and input parameters are well documented with clear descriptions. The optional version input with fallback logic and dry run capability provide good flexibility.


40-53: LGTM! Well-configured publish job setup.

The publish job is properly configured with build dependency and environment settings.

.github/workflows/publish_sdm_connector.yml Outdated Show resolved Hide resolved
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (5)
.github/workflows/publish_sdm_connector.yml (5)

9-16: Consider adding pattern validation for version input?

The version input could benefit from a pattern validation to ensure it matches semantic versioning format. wdyt about adding something like this?

   version:
     description:
         The version to publish, ie 1.0.0 or 1.0.0-dev1.
         If omitted, and if run from a release branch, the version will be
         inferred from the git tag.
         If omitted, and if run from a non-release branch, then only a SHA-based
         Docker tag will be created.
     required: false
+    pattern: '^[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9]+)?$'

24-39: Add caching to speed up builds?

The build job could be faster with dependency caching. How about adding the Python cache action? Something like:

   build:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+      - uses: actions/cache@v3
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-

92-96: Consider more comprehensive prerelease detection?

The current regex only considers exact semver (X.Y.Z) as non-prerelease. Should we also handle common prerelease indicators? Something like:

-          if [[ "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+          if [[ "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] && ! [[ "${VERSION}" =~ (rc|alpha|beta|dev) ]]; then

116-130: Consider using jq for more robust tag checking?

The current tag checking might fail with rate limits. How about using the Docker Hub API with jq? Something like:

       - name: Check for existing tag
         if: env.VERSION != ''
         run: |
           tag="airbyte/source-declarative-manifest:${{ env.VERSION }}"
-          if [ -z "$tag" ]; then
-            echo "Error: VERSION is not set. Ensure the tag follows the format 'refs/tags/vX.Y.Z'."
-            exit 1
-          fi
           echo "Checking if tag '$tag' exists on DockerHub..."
-          if DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect "$tag" > /dev/null 2>&1; then
+          if curl -s "https://hub.docker.com/v2/repositories/airbyte/source-declarative-manifest/tags/${VERSION}" | jq -e '.name' > /dev/null; then
             echo "The tag '$tag' already exists on DockerHub. Skipping publish to prevent overwrite."
             exit 1
           fi

131-163: Consider consolidating build steps?

We have three very similar build steps that only differ in tags. Would it be cleaner to consolidate them? Something like:

-      - name: Build and push (sha tag)
-        if: env.VERSION == '' && github.event.inputs.dry_run == 'false'
-        uses: docker/build-push-action@v5
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64
-          push: true
-          tags: |
-            airbyte/source-declarative-manifest:${{ github.sha }}
-
-      - name: Build and push (version tag)
-        if: env.VERSION != '' && github.event.inputs.dry_run == 'false'
-        uses: docker/build-push-action@v5
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64
-          push: true
-          tags: |
-            airbyte/source-declarative-manifest:${{ env.VERSION }}
-
-      - name: Build and push ('latest' tag)
-        if: env.VERSION != '' && env.IS_PRERELEASE == 'false' && github.event.inputs.dry_run == 'false'
-        uses: docker/build-push-action@v5
-        with:
-          context: .
-          platforms: linux/amd64,linux/arm64
-          push: true
-          tags: |
-            airbyte/source-declarative-manifest:latest
+      - name: Prepare tags
+        id: prep
+        run: |
+          TAGS="airbyte/source-declarative-manifest:${{ github.sha }}"
+          if [ -n "$VERSION" ]; then
+            TAGS="$TAGS,airbyte/source-declarative-manifest:${{ env.VERSION }}"
+            if [ "$IS_PRERELEASE" = "false" ]; then
+              TAGS="$TAGS,airbyte/source-declarative-manifest:latest"
+            fi
+          fi
+          echo "tags=${TAGS}" >> $GITHUB_OUTPUT
+
+      - name: Build and push
+        if: github.event.inputs.dry_run == 'false'
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: ${{ steps.prep.outputs.tags }}

This would make the workflow more maintainable and reduce duplication. wdyt?

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 953e66d and a9da960.

📒 Files selected for processing (1)
  • .github/workflows/publish_sdm_connector.yml (1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.3)
.github/workflows/publish_sdm_connector.yml

56-56: shellcheck reported issue in this script: SC2086:info:6:48: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:6:24: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:24:30: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:27:33: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:29:32: Double quote to prevent globbing and word splitting

(shellcheck)

🔇 Additional comments (2)
.github/workflows/publish_sdm_connector.yml (2)

40-53: LGTM! Clean job setup with proper conditions and environment.


97-103: LGTM! Proper artifact handling between jobs.

.github/workflows/publish_sdm_connector.yml Outdated Show resolved Hide resolved
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (4)
.github/workflows/publish_sdm_connector.yml (4)

9-16: How about adding more specific version format requirements to the description? wdyt?

The description could be more explicit about the expected version format. Maybe something like:

        description: |
-           The version to publish, ie 1.0.0 or 1.0.0-dev1.
+           The version to publish (e.g., '1.0.0' for releases or '1.0.0-dev1' for prereleases).
+           Must follow semantic versioning format: MAJOR.MINOR.PATCH[-PRERELEASE].
            If omitted, and if run from a release branch, the version will be
            inferred from the git tag.
            If omitted, and if run from a non-release branch, then only a SHA-based
            Docker tag will be created.

45-47: Consider using a reusable environment URL? wdyt?

Instead of hardcoding the DockerHub URL, we could make it more maintainable by using a repository variable. Something like:

    environment:
      name: DockerHub
-     url: https://hub.docker.com/r/airbyte/source-declarative-manifest/tags
+     url: ${{ vars.DOCKER_HUB_TAGS_URL }}

116-130: Consider enhancing the tag check with rate limiting protection? wdyt?

The DockerHub API has rate limits. Maybe we could add some retry logic:

        run: |
          tag="airbyte/source-declarative-manifest:${{ env.VERSION }}"
          if [ -z "$tag" ]; then
            echo "Error: VERSION is not set. Ensure the tag follows the format 'refs/tags/vX.Y.Z'."
            exit 1
          fi
          echo "Checking if tag '$tag' exists on DockerHub..."
+         max_attempts=3
+         attempt=1
+         while [ $attempt -le $max_attempts ]; do
+           echo "Attempt $attempt of $max_attempts"
            if DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect "$tag" > /dev/null 2>&1; then
              echo "The tag '$tag' already exists on DockerHub. Skipping publish to prevent overwrite."
              exit 1
+           elif [ $? -eq 1 ]; then
+             echo "No existing tag '$tag' found. Proceeding with publish."
+             exit 0
+           else
+             echo "Rate limit or network error. Waiting before retry..."
+             sleep $((attempt * 5))
+             attempt=$((attempt + 1))
+           fi
+         done
-           echo "No existing tag '$tag' found. Proceeding with publish."
+         echo "Error: Failed to check tag after $max_attempts attempts"
+         exit 1

131-163: How about adding Docker layer caching to speed up builds? wdyt?

The build steps could benefit from caching. Consider adding these options:

        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
+         cache-from: type=gha
+         cache-to: type=gha,mode=max
          tags: |
            airbyte/source-declarative-manifest:${{ env.VERSION }}
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between a9da960 and 7763245.

📒 Files selected for processing (1)
  • .github/workflows/publish_sdm_connector.yml (1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.3)
.github/workflows/publish_sdm_connector.yml

56-56: shellcheck reported issue in this script: SC2086:info:6:48: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:6:24: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:24:30: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:27:33: Double quote to prevent globbing and word splitting

(shellcheck)


66-66: shellcheck reported issue in this script: SC2086:info:29:32: Double quote to prevent globbing and word splitting

(shellcheck)

🔇 Additional comments (2)
.github/workflows/publish_sdm_connector.yml (2)

24-39: LGTM! Clean and efficient build setup.

The build job uses the latest action versions and properly handles artifact paths.


98-102: LGTM! Proper artifact handling.

The artifact download step is well-configured and uses the latest action version.

.github/workflows/publish_sdm_connector.yml Outdated Show resolved Hide resolved
@aaronsteers
Copy link
Contributor Author

Findings:

  • New AIRBYTE_ENTRYPOINT env var needs to be "source-declarative-manifest" without "poerty run". This is because Orchestrator is overiding working directory.
  • Actual entrypoint should match for consistency during testing.

@ChristoGrab ChristoGrab self-requested a review November 22, 2024 19:00
Copy link
Collaborator

@ChristoGrab ChristoGrab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Will follow up with the update to the Dockerfile

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (2)
.github/workflows/publish_sdm_connector.yml (2)

9-16: How about adding version format validation to the input? wdyt?

The version input description is clear, but we could add a pattern to validate the format upfront. This would catch invalid versions earlier in the process.

       version:
         description:
             The version to publish, ie 1.0.0 or 1.0.0-dev1.
             If omitted, and if run from a release branch, the version will be
             inferred from the git tag.
             If omitted, and if run from a non-release branch, then only a SHA-based
             Docker tag will be created.
         required: false
+        pattern: '^v?\d+\.\d+\.\d+(-[a-zA-Z0-9]+)?$'

143-164: How about optimizing the Docker builds? wdyt?

We could improve build performance by:

  1. Adding build caching
  2. Making platforms configurable
       - name: Build and push (sha tag)
         if: env.VERSION == '' && github.event.inputs.dry_run == 'false'
         uses: docker/build-push-action@v5
         with:
           context: .
-          platforms: linux/amd64,linux/arm64
+          platforms: ${{ inputs.platforms || 'linux/amd64,linux/arm64' }}
           push: true
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
           tags: |
             airbyte/source-declarative-manifest:${{ github.sha }}

Would you like me to add a new workflow input for configuring build platforms?

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 7763245 and 73f2c1c.

📒 Files selected for processing (1)
  • .github/workflows/publish_sdm_connector.yml (1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.3)
.github/workflows/publish_sdm_connector.yml

30-30: shellcheck reported issue in this script: SC2086:info:6:48: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:6:24: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:24:30: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:25:30: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:28:33: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:29:33: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:31:32: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:32:32: Double quote to prevent globbing and word splitting

(shellcheck)


101-101: property "version" is not defined in object type {}

(expression)


102-102: property "is_prerelease" is not defined in object type {}

(expression)

.github/workflows/publish_sdm_connector.yml Show resolved Hide resolved
.github/workflows/publish_sdm_connector.yml Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
.github/workflows/publish_sdm_connector.yml (2)

9-16: How about enhancing the version input description? wdyt?

The description could be more explicit about version format requirements. Consider:

       version:
         description:
-            The version to publish, ie 1.0.0 or 1.0.0-dev1.
+            The version to publish (e.g., '1.0.0' or '1.0.0-dev1').
+            Must follow semantic versioning format (X.Y.Z[-suffix]).
             If omitted, and if run from a release branch, the version will be
             inferred from the git tag.
             If omitted, and if run from a non-release branch, then only a SHA-based
             Docker tag will be created.

147-154: How about adding Docker layer caching to speed up builds? wdyt?

The build steps could benefit from Docker layer caching to improve build times. Consider:

       uses: docker/build-push-action@v5
       with:
         context: .
         platforms: linux/amd64,linux/arm64
         push: true
+        cache-from: type=gha
+        cache-to: type=gha,mode=max
         tags: |

Also applies to: 158-165, 170-176

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 73f2c1c and 832c57c.

📒 Files selected for processing (1)
  • .github/workflows/publish_sdm_connector.yml (1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.3)
.github/workflows/publish_sdm_connector.yml

30-30: shellcheck reported issue in this script: SC2086:info:6:48: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:6:24: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:24:30: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:25:30: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:28:33: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:29:33: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:31:32: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:32:32: Double quote to prevent globbing and word splitting

(shellcheck)


102-102: property "version" is not defined in object type {}

(expression)


103-103: property "is_prerelease" is not defined in object type {}

(expression)

🔇 Additional comments (2)
.github/workflows/publish_sdm_connector.yml (2)

1-176: LGTM! The workflow is well-structured and robust.

The workflow effectively handles:

  • Version validation and management
  • Multi-platform builds
  • Prerelease version detection
  • Existing tag checks
🧰 Tools
🪛 actionlint (1.7.3)

30-30: shellcheck reported issue in this script: SC2086:info:6:48: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:6:24: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:24:30: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:25:30: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:28:33: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:29:33: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:31:32: Double quote to prevent globbing and word splitting

(shellcheck)


40-40: shellcheck reported issue in this script: SC2086:info:32:32: Double quote to prevent globbing and word splitting

(shellcheck)


102-102: property "version" is not defined in object type {}

(expression)


103-103: property "is_prerelease" is not defined in object type {}

(expression)


98-100: Should we verify the DockerHub environment configuration? wdyt?

Please ensure that the DockerHub environment is properly configured in GitHub with the required secrets and any necessary approval processes or protection rules.

@aaronsteers aaronsteers enabled auto-merge (squash) November 22, 2024 19:32
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (4)
.github/workflows/publish_sdm_connector.yml (4)

9-16: How about enhancing the version input description? wdyt?

The description could be more explicit about the expected version format. Consider:

       version:
         description:
-            The version to publish, ie 1.0.0 or 1.0.0-dev1.
+            The version to publish (e.g., '1.0.0' or '1.0.0-dev1').
+            Must follow semantic versioning format: X.Y.Z[-suffix].
             If omitted, and if run from a release branch, the version will be
             inferred from the git tag.
             If omitted, and if run from a non-release branch, then only a SHA-based
             Docker tag will be created.

68-74: Should we make the prerelease version detection more robust? wdyt?

The current regex only matches exact semver without prerelease tags. Consider adding validation for the prerelease suffix format:

-          if [[ "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+          if [[ "${VERSION}" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$ ]]; then
             echo "IS_PRERELEASE=false" >> $GITHUB_ENV
             echo "IS_PRERELEASE=false" >> $GITHUB_OUTPUT
           else
             echo "IS_PRERELEASE=true" >> $GITHUB_ENV
             echo "IS_PRERELEASE=true" >> $GITHUB_OUTPUT
           fi

This would ensure that prerelease suffixes follow the semver spec (e.g., -alpha.1, -beta.2, -rc.1).


153-154: Should we optimize the Docker build context? wdyt?

The build context includes the entire repository, which might slow down builds. Consider:

  1. Adding .dockerignore to exclude unnecessary files
  2. Using a specific subdirectory as context if possible

Example .dockerignore:

.git/
.github/
tests/
*.md
*.pyc
__pycache__/

Also applies to: 164-165, 176-177


128-132: Should we consider using short-lived Docker Hub tokens? wdyt?

For enhanced security, consider:

  1. Using short-lived access tokens instead of a password
  2. Setting token expiration to match your release cycle
  3. Adding token scope restrictions to only allow push access

This helps minimize the impact if credentials are ever compromised.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 832c57c and 975b396.

📒 Files selected for processing (1)
  • .github/workflows/publish_sdm_connector.yml (1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.3)
.github/workflows/publish_sdm_connector.yml

30-30: shellcheck reported issue in this script: SC2086:info:6:48: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:6:24: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:24:30: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:25:30: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:28:33: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:29:33: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:31:32: Double quote to prevent globbing and word splitting

(shellcheck)


41-41: shellcheck reported issue in this script: SC2086:info:32:32: Double quote to prevent globbing and word splitting

(shellcheck)

@aaronsteers aaronsteers disabled auto-merge November 22, 2024 19:38
@aaronsteers aaronsteers merged commit e52bde4 into main Nov 22, 2024
20 of 21 checks passed
@aaronsteers aaronsteers deleted the aj/chore/move-sdm-publish-to-dedicated-step branch November 22, 2024 19:38
@aaronsteers aaronsteers mentioned this pull request Nov 22, 2024
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants