Merge branch 'fluxcd:main' into main

.github/workflows/check.yaml

@@ -11,9 +11,9 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
+        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
       - name: Setup Terraform
-        uses: hashicorp/setup-terraform@97f030cf6dc0b4f5e0da352c7bca9cca34579800 # v2
+        uses: hashicorp/setup-terraform@651471c36a6092792c552e8b1bef71e592b462d8 # v2
         with:
           terraform_version: latest
           terraform_wrapper: false

.github/workflows/sync-labels.yaml

@@ -17,7 +17,7 @@ jobs:
     permissions:
       issues: write
     steps:
-      - uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
+      - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
       - uses: EndBug/label-sync@52074158190acb45f3077f9099fea818aa43f97a # v2.3.3
         with:
           # Configuration file

.gitignore

@@ -0,0 +1,4 @@
+.env
+.terraform
+.terraform.*
+terraform.tfstate*

accounts/aws/README.md

@@ -0,0 +1,76 @@
+# AWS Account
+
+This documents how to set up an AWS account, prepare it to use for Flux test
+infrastructure and various usage workflows for managing the account.
+
+## New account initial setup
+
+- Once a new AWS account is created, log in as the root user and enable
+  multi-factor authentication for the root account, refer
+  https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_enable.html.
+- For Billing and Cost Management in the account, enable IAM access to billing,
+  refer https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_billing.html.
+  This will enable the other users to be able to view the billing console if
+  they have the necessary permissions. With access to billing console, the users
+  would be able to know the cost of their resource usage and help keep the cost
+  in control.
+- For user management IAM Identity Center is used, which makes it easy to invite
+  and manage users in the account. AWS sends invitation email to the users and
+  provides an access portal to verify the email address, set up MFA device and
+  help log into the account easily. Choose a region, say `us-east-2`, switch to
+  the region in the AWS web console and enable IAM Identity Center, refer
+  https://docs.aws.amazon.com/SetUp/latest/UserGuide/setup-enableIdC.html. If
+  asked, create an AWS Organization and enable the Identity Center as an
+  organization, which is the recommended usage by AWS.
+  - After enabling IAM Identity Center, go to the IAM Identity Center console
+    settings and enable Multi-factor authentication, refer
+    https://docs.aws.amazon.com/singlesignon/latest/userguide/mfa-getting-started.html.
+    Configure the following options:
+      - Under **Prompt users for MFA**, select *Only when the sign-in context
+        changes*.
+      - Under **Users can authenticate with these MFA types**, select both
+        *Security keys* and *Authenticator apps*.
+      - Under **If a user does not yet have a registered MFA device**, select
+        *Require them to register an MFA device at sign in*.
+  - Under the **Authentication** tab in IAM Identity Center settings, configure
+    the **Standard authentication** to *Send email OTP for users created from
+    API*. This will make sure invitation emails are sent to the users when
+    created using terraform or other tooling.
+- Some tools like aws-nuke require the AWS account to have an alias set before
+  operating on the account. Set an account alias in the IAM Dashboard, refer
+  https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html.
+  It can be set to `fluxcd` or edited if needed in the future.
+
+The above covers the initial setup. Further account setup will be done by code
+as described in the following sections.
+
+## Account management
+
+After the initial setup, the account can be managed using terraform
+configurations for provisioning and maintaining all the resources.
+
+`main.tf` contains terraform configuration for creating IAM Identity Center
+permission sets, groups using the permission sets, their association with the
+AWS account, users for web console access, IAM Identity providers which are used
+in the tests for authenticating with federated identities and assuming roles
+with permissions needed for running the tests.
+
+For first time setup, an IAM user can be created manually with the administrator
+access to apply the configurations in `main.tf`. This will create the user
+accounts who can log in and use the account.
+
+**NOTE:** Due to a limitation in the AWS Identity Center API, the user accounts
+created via API require explicit email verification. Refer
+https://github.com/hashicorp/terraform-provider-aws/issues/28102 for details.
+Due to this, after creating a new user, an administrator needs to go to the
+user's page and click on **Send email verification link** button.
+
+After applying the configuration, the IAM user can be deleted and the created
+non-root users can be used to manage the account. Also see
+https://docs.aws.amazon.com/signin/latest/userguide/iam-id-center-sign-in-tutorial.html
+for details about AWS access portal sign in.
+
+The account can be managed by updating the terraform code and regularly applying
+the changes using terraform in a GitHub actions workflow. Updates to users and
+resources in the account can be go through the usual GitHub pull request
+workflow.

accounts/aws/main.tf

@@ -0,0 +1,83 @@
+data "aws_caller_identity" "current" {}
+data "aws_ssoadmin_instances" "current" {}
+
+# Create a permission set for administrator access.
+resource "aws_ssoadmin_permission_set" "admin" {
+  name             = "AdministratorAccess"
+  instance_arn     = tolist(data.aws_ssoadmin_instances.current.arns)[0]
+  description      = "To be used to grant administrator access to users and groups."
+  session_duration = "PT8H"
+  # TODO: Decide and add tags.
+}
+
+# Create a group for administrators.
+resource "aws_identitystore_group" "admin" {
+  identity_store_id = tolist(data.aws_ssoadmin_instances.current.identity_store_ids)[0]
+  display_name      = "Admin"
+  description       = "Admin Group"
+}
+
+# Assign the admin group and permission set.
+resource "aws_ssoadmin_account_assignment" "admin_account_assignment" {
+  instance_arn       = tolist(data.aws_ssoadmin_instances.current.arns)[0]
+  permission_set_arn = aws_ssoadmin_permission_set.admin.arn
+
+  principal_id   = aws_identitystore_group.admin.group_id
+  principal_type = "GROUP"
+
+  target_id   = data.aws_caller_identity.current.account_id
+  target_type = "AWS_ACCOUNT"
+}
+
+# Attach a AdministratorAccess managed policy to the administrator permission
+# set.
+# NOTE: Since this attachment affects accounts the permission set is associated
+# with, it has to depends on the account assignment.
+resource "aws_ssoadmin_managed_policy_attachment" "admin" {
+  depends_on = [aws_ssoadmin_account_assignment.admin_account_assignment]
+
+  instance_arn       = tolist(data.aws_ssoadmin_instances.current.arns)[0]
+  managed_policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
+  permission_set_arn = aws_ssoadmin_permission_set.admin.arn
+}
+
+# Create user and assign a group.
+resource "aws_identitystore_user" "darkowlzz" {
+  identity_store_id = tolist(data.aws_ssoadmin_instances.current.identity_store_ids)[0]
+
+  display_name = "Sunny"
+  user_name    = "darkowlzz"
+
+  name {
+    given_name  = "Sunny"
+    family_name = "Sunny"
+  }
+
+  emails {
+    value   = "[email protected]"
+    primary = true
+  }
+}
+resource "aws_identitystore_group_membership" "darkowlzz_admin" {
+  identity_store_id = tolist(data.aws_ssoadmin_instances.current.identity_store_ids)[0]
+  group_id          = aws_identitystore_group.admin.group_id
+  member_id         = aws_identitystore_user.darkowlzz.user_id
+}
+
+# Register GitHub OIDC identity provider.
+resource "aws_iam_openid_connect_provider" "github" {
+  url = "https://token.actions.githubusercontent.com"
+
+  client_id_list = [
+    "sts.amazonaws.com",
+  ]
+
+  # For obtaining the thumbprint, refer
+  # https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html.
+  # Another easier way to obtain this is from the AWS IAM Identity Provider web
+  # console. When a provider is added through the web console, the thumbprint is
+  # optional. AWS automatically obtains it and shows it in the console.
+  thumbprint_list = ["1b511abead59c6ce207077c0bf0e0043b1382612"]
+
+  # TODO: Decide and add tags.
+}

tf-modules/aws/github-actions/README.md

@@ -0,0 +1,118 @@
+# AWS GitHub Actions Secrets and Variables
+
+This terraform module creates AWS policy and role to be used in GitHub actions
+by assuming the created role with OIDC federation. The GitHub action assumes the
+AWS role by authenticating via GitHub OpenID Connect (OIDC) identity provider,
+refer [Use IAM roles to connect GitHub Actions to actions in
+AWS](https://aws.amazon.com/blogs/security/use-iam-roles-to-connect-github-actions-to-actions-in-aws/).
+This can be made easy by using [Configure AWS
+Credentials](https://github.com/marketplace/actions/configure-aws-credentials-action-for-github-actions)
+GitHub action.
+
+By default, the following GitHub actions secrets are created:
+- `AWS_ACCOUNT_ID`
+- `AWS_ASSUME_ROLE_NAME`
+
+and `AWS_REGION` actions variable is created. All these names are
+overridable, see `variables.tf`.
+
+It also supports adding custom secrets and variables in addition to the above.
+
+**NOTE:** Overwriting existing GitHub secrets and variables is not supported.
+
+## Usage
+
+```hcl
+module "aws_gh_actions" {
+  source = "git::https://github.com/fluxcd/test-infra.git//tf-modules/aws/github-actions"
+
+  aws_policy_name        = "test-policy-1"
+  aws_policy_description = "For running e2e tests"
+  aws_provision_perms    = [
+    "ec2:CreateInternetGateway",
+    "ec2:CreateLaunchTemplate",
+    "ec2:CreateLaunchTemplateVersion",
+  ]
+  aws_cluster_role_prefix    = [
+    "flux-test-",
+    "blue-eks-node-group-",
+    "green-eks-node-group-"
+  ]
+  aws_role_name          = "test-role-1"
+  aws_role_description   = "Role to be assumed by github actions"
+  github_repo_owner      = "fluxcd"
+  github_project         = "repo-name"
+  github_repo_branch_ref = "ref:refs/heads/main"
+
+  github_variable_custom = {
+      "SOME_VAR1" = "some-val1",
+      "SOME_var2" = "some-val2"
+  }
+  github_secret_custom = {
+      "SECRET1" = "some-secret1",
+      "SECRET2" = "some-secret2"
+  }
+}
+```
+
+## AWS Requirements
+
+Use the following IAM policy document to grant the needed permissions.
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Sid": "VisualEditor0",
+            "Effect": "Allow",
+            "Action": [
+                "iam:AttachRolePolicy",
+                "iam:CreatePolicy",
+                "iam:CreatePolicyVersion",
+                "iam:CreateRole",
+                "iam:DeletePolicy",
+                "iam:DeletePolicyVersion",
+                "iam:DeleteRole",
+                "iam:DetachRolePolicy",
+                "iam:GetPolicy",
+                "iam:GetPolicyVersion",
+                "iam:GetRole",
+                "iam:ListAttachedRolePolicies",
+                "iam:ListInstanceProfilesForRole",
+                "iam:ListPolicyVersions",
+                "iam:ListRolePolicies"
+            ],
+            "Resource": "*"
+        }
+    ]
+}
+```
+
+Since the GitHub actions use GitHub OIDC identity provider, the AWS account must
+have GitHub as an existing identity provider, see [Configuring OpenID Connect in
+Amazon Web
+Services](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services).
+The provider URL is expected to be `https://token.actions.githubusercontent.com`
+and the audience `sts.amazonaws.com`, as an account can only have a single
+instance of this identity provider. These are hard-coded in the configurations
+and should be updated in the source, if needed.
+
+## GitHub Requirements
+
+Create a GitHub fine-grained token for the target repository with the following
+repository permissions:
+- `Read access to metadata`
+- `Read and Write access to actions variables and secrets`
+
+## Provider Configuration
+
+Configure the AWS and GitHub provider with the following environment variables:
+```sh
+export AWS_ACCESS_KEY_ID=""
+export AWS_SECRET_ACCESS_KEY=""
+
+export GITHUB_TOKEN=""
+```
+
+Check the respective provider docs for more details.