Use configure-aws-credentials workflow instead of passing secret_access_key

[vudiep411]

Summary

This PR fixes #1346 where we can get rid of the long term credentials by using OpenID Connect. OpenID Connect (OIDC) allows your GitHub Actions workflows to access resources in Amazon Web Services (AWS), without needing to store the AWS credentials as long-lived GitHub secrets.

Changes

We can remove these secrets that were passed in previously:

token: ${{ secrets.GITHUB_TOKEN }}
      bucket: ${{ secrets.AWS_S3_BUCKET }}
      access_key_id: ${{ secrets.AWS_S3_ACCESS_KEY_ID }}
      secret_access_key: ${{ secrets.AWS_S3_ACCESS_KEY }}

Instead we only need the role-to-assume arn. For more information OIDC.

Prerequisites

Before merging this PR, we need to make sure to set up the proper Identity providers on the production AWS account. Follow this guides.
Quick guide:

1. Create an Identity provider with OpenID Connect.
   Provider url: https://token.actions.githubusercontent.com
   
   Audience: sts.amazonaws.com

2. Assign the role into this new provider with this trust policy like below

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<aws_account_id>:oidc-provider/token.actions.githubusercontent.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringLike": {
                    "token.actions.githubusercontent.com:sub": [
                        "repo:valkey-io/valkey:ref:refs/heads/unstable",
                        "repo:valkey-io/valkey:ref:refs/tags/*"
                    ],
                    "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}

3. In Github secrets, Add these secrets:

Results

Github action run:

Files in S3 Dev env:

Note: the failed workflow can be found in this issue #1343 which will be in a separate PR.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.84%. Comparing base (9305b49) to head (1d97e1c).
Report is 20 commits behind head on unstable.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #1363      +/-   ##
============================================
+ Coverage     70.62%   70.84%   +0.22%     
============================================
  Files           117      118       +1     
  Lines         63315    63561     +246     
============================================
+ Hits          44714    45030     +316     
+ Misses        18601    18531      -70     

see 27 files with indirect coverage changes

[enjoy-binbin]

seems ok, @roshkhatri please take a look

[roshkhatri]

LGTM, just had a few questions.

[roshkhatri]

We will also need to do the prerequisites steps for the main repo before we merge this.

[roshkhatri]

@vudiep411, will it be possible to add a test, where we uploads a test build binary to a test s3 bucket when ever changes are made to these workflows.

With this we can be sure that when we release stuff, it doesn't break on the main valkey repository

[vudiep411]

Yes it is possible, we can use github environment for that. So we can write something like this:

- name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ env.ROLE_TO_ASSUME }}
          role-session-name: ${{ env.ROLE_SESSION_NAME }}
          aws-region: ${{ env.AWS_REGION }}

We use env to dynamically assigned the role to assume and env. But this would required us to set up multiple OCID on different accounts not just in the workflow itself so it should be in a separate issue. For now we can just set this one up and after this looks good, I will open another issue for it if that's ok.
@roshkhatri

[roshkhatri]

I think we can also do it based on the github event trigger maybe?

1. We set two OCID for two buckets, releases and test.
2. When the changes are made to release workflow files, for a pull_request event we run the same workflow with secrets.AWS_TEST_ROLE_TO_ASSUME and secrets.AWS_TEST_S3_BUCKET and for a workflow_dispatch and release
   event trigger we run with the current setup.
3. We would do it at the head of unstable.
4. Once we upload the test binaries, we can confirm the upload, delete the delete the file and pass the test.

Also, I think we should add the test on the same PR so we can run the test on this PR and know that the tests also work. We would also have to do the pre-requisites only once for both the scenarios.

[vudiep411]

That's a really good suggestion. We can look into that. That way it would automated the testing of a PR with a test bucket. Definitely doable

[vudiep411]

New Changes

1. When a PR is raised, the build-release will run if there are changes to the release workflow or utils/releasetools
2. Get the version step will output the version that is getting built. eg. 8.0.0, unstable,...
3. Set bucket name test will output true if the event is a PR else it will output false. This step is to determine if we want to use the test bucket or prod bucket to store the binary files.

Set up

Trust policy for github-action-role

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Principal": {
				"Federated": "arn:aws:iam::<aws_account_number>:oidc-provider/token.actions.githubusercontent.com"
			},
			"Action": "sts:AssumeRoleWithWebIdentity",
			"Condition": {
				"StringLike": {
					"token.actions.githubusercontent.com:sub": [
						"repo:valkey/valkey:ref:refs/heads/*",
						"repo:vudiep411/valkey:ref:refs/tags/*",
						"repo:vudiep411/valkey:pull_request"
					],
					"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
				}
			}
		}
	]
}

Repo Secrets
AWS_ROLE_TO_ASSUME: ARN role created from the prerequisite
AWS_S3_BUCKET: Prod bucket name
AWS_TEST_BUCKET: Test bucket name

Results

• This is the PR replicated in my env.
• Check the CI build-release for this PR here

AWS Buckets
We will have 2 buckets. 1 is a test bucket and the other is a prod bucket

Summary

The PR CI build-release will build the binary files into the test bucket to check if these files are correctly build. After that we can merge the PR and whenever we release a new version, this workflow will run and build the binary files into the production bucket.

@roshkhatri

[roshkhatri]

LGTM

[roshkhatri]

LGTM! once this PR is approve, we can make the necessary changes to the prerequisite AWS configs and merge this one in.

Can you also update the top comment to mention all the changes we made here later
such as, adding tests, etc.