Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate deployment #163

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

shanice-skylight
Copy link
Collaborator

@shanice-skylight shanice-skylight commented Nov 15, 2024

PULL REQUEST

Summary

The full_cd.yaml file creates a workflow the ties together all of the actions and workflows to automate the deployment of the new image created by this repository to EKS. See the workflow details below:

  1. Build the image and push it to the dibbs-query-connector repository's registry.
  2. Trigger the workflow in the phdi-charts repository to update the chart version by creating a PR that automatically modifies the chart file version.
  3. Manually approve and merge the PR due to branch protection rules on the main branch, which require approval before merging.
  4. Trigger the workflow in the phdi-playground repository to deploy the new image created in Step 1.
  5. Pull the latest image from the GHCR in the dibbs-query-connector repository and deploy it to EKS.

Related Issue

Fixes # Automates most of the manual steps that were required in order to deploy a new image to the demo site.

Additional Information

Original Deployment
Note: Manual means the developer has to go to Github actions to trigger the workflow
QC Deployment Workflow-Original State drawio (2)

Current Deployment with this PR
Note: This deployment would involve changing the individual workflows in the current deployment into actions that will be compiled together into one workflow and wait until the previous action is complete before moving to the next action.
QC Deployment Workflow-End State v1 3 drawio

Interdependent on the following PR:
- phdi-charts

Checklist

  • [X ] Descriptive Pull Request title
  • Link to relevant issues
  • Update documentation

@shanice-skylight shanice-skylight self-assigned this Nov 15, 2024
@shanice-skylight shanice-skylight marked this pull request as draft November 19, 2024 18:32
@shanice-skylight shanice-skylight requested review from nickclyde and removed request for rin-skylight November 23, 2024 00:12
Copy link
Collaborator

@rin-skylight rin-skylight left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great start, but I'd recommend doing some cleanup and tailoring before this is ready to go.

tags: |
ghcr.io/${{ env.REPO }}/query-connector:main, ghcr.io/${{ env.REPO }}/query-connector:latest

# - name: Output image digest
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this commented code needed? If not, consider deleting.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

context: ./query-connector
push: true
tags: |
ghcr.io/${{ env.REPO }}/query-connector:main, ghcr.io/${{ env.REPO }}/query-connector:latest
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're going to need some sort of hash or semantic versioning to be added here. main and latest will not reliably trigger cloud platforms to pull and deploy a new image if one already exists with these tags.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition, does env.REPO contain the CDCgov part of your tag? Again, I recommend simplifying this.

Copy link
Collaborator Author

@shanice-skylight shanice-skylight Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For triggering the new image to be pulled and deployed, the up_helm_chart job triggers the workflow in phdi-charts to create a PR that increases the chart version. I linked the PR that adds this workflow in phdi-charts. Then the workflow waits until this step is completed before attempting to pull the new image and deploy it in the build and push step.

I'm not oppose to adding the semantic versioning or a hash at the end. Am I right in assuming we would not have to increase the chart version if we went this route?

For the env.REPO, it returns cdcgov/dibbs-query-connector which evaluates to the full path ghcr.io/cdcgov/dibbs-query-connector/query-connector:latest`, this links to the path configured in phdi-charts

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified the tag and the path in a PR for phdi-charts

name: Poll for status of workflow
description: This action will wait and poll for the status of workflow

# on:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More code comments. Clean these up, please.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is no longer being used due to the built in github keyword needs being used in its placed. Removed action from code.

if: github.event_name == 'push'
id: version
run: |
# Check if there's an existing version file; if not, start from 1.0.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't recommend doing things this way. If the version file doesn't exist, I would recommend failing the deployment outright.

Reason: Assume someone has deleted the version file in their commit on accident. You run this, and create a new 1.0.0 image. You're actually on 5.0.6. You've just overwritten your archived 1.0.0 image that someone in production may have been depending on.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, I modified the bash script to add an error message and fail the step if the version file does not exist.

env:
REPO: ${{ github.repository }}
run: |
echo "REPO=${GITHUB_REPOSITORY,,}" >> $GITHUB_ENV
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question! Any particular reason why we opted to make this into a variable? Theoretically, this should never change. This seems like it's a little overkill.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming whomever initially created the CD workflow added it so it would align with the value configured in phdi-charts.

I can modify it and create a PR in phdi-charts to change this as well

@@ -0,0 +1,58 @@
name: Poll for status of workflow
description: This action will wait and poll for the status of workflow
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you fill me in on what the purpose of this is, or why it's needed? Is this for checking the status of a remote workflow in another repo? Also, are there potential issues with this approach if the scope of the PAT changes?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is no longer being used due to the built in github keyword needs being used in its placed. Removed action from code.

if: github.event_name == 'push'
id: version
run: |
# Check if there's an existing version file; if not, start from 1.0.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments below regarding version files.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified the bash script to add an error message and fail the step if the version file does not exist.

push:
branches:
- main
- shanice/automated-workflow-deployment
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is your personal branch intended to be here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used for testing purposes, removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants