Skip to content

Commit

Permalink
Merge pull request #39 from fluxcd/reaper-aws-nuke
Browse files Browse the repository at this point in the history 
reaper: Add aws-nuke integration
  • Loading branch information
@darkowlzz
darkowlzz authored Jul 31, 2024
2 parents c6db8ab + aa4fb44 commit c8e430d
Show file tree
Hide file tree
Showing 12 changed files with 1,705 additions and 41 deletions.
89 changes: 78 additions & 11 deletions tools/reaper/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ For listing the resources, readonly access to all the resources is needed.
- AWS: Use the builtin `AWSResourceGroupsReadOnlyAccess` IAM policy .
- Azure: Use the builtin `Reader` IAM role.
- GCP: Use the builtin `Cloud Asset Viewer` IAM role.
- aws-nuke: See below for an AWS IAM policy document.

For deleting the resources, grant the delete permission for the individual
resources.
Expand All @@ -25,6 +26,77 @@ account with the following permissions to delete integration test resources:
- `artifactregistry.repositories.get`
- `artifactregistry.repositories.delete`

For [aws-nuke][aws-nuke], a new deleter IAM policy
can be created and assigned to the reaper IAM principal with the following
policy document:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "reaper",
"Effect": "Allow",
"Action": [
"iam:ListPolicies",
"iam:ListRoles",
"iam:ListOpenIDConnectProviders",
"iam:ListAttachedRolePolicies",
"iam:ListAccountAliases",
"iam:ListRolePolicies",
"iam:GetRole",
"iam:GetPolicy",
"iam:GetOpenIDConnectProvider",
"ec2:DescribeAddresses",
"ec2:DescribeInstances",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeNatGateways",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeInternetGateways",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeVpcs",
"ec2:DescribeVolumes",
"ec2:DescribeSubnets",
"ec2:DescribeRouteTables",
"autoscaling:DescribeAutoScalingGroups",
"eks:ListClusters",
"eks:ListNodegroups",
"eks:DescribeCluster",
"eks:DescribeNodegroup",
"ecr:ListTagsForResource",
"ecr:DescribeRepositories",
"ec2:DeleteSubnet",
"ec2:DeleteRouteTable",
"ec2:DeleteVolume",
"ec2:DeleteTags",
"ec2:DeleteInternetGateway",
"ec2:DetachInternetGateway",
"ec2:RevokeSecurityGroupEgress",
"ec2:RevokeSecurityGroupIngress",
"ec2:DeleteSecurityGroup",
"ec2:DeleteNatGateway",
"ec2:DeleteVpc",
"ec2:ReleaseAddress",
"ec2:DeleteLaunchTemplate",
"ec2:TerminateInstances",
"eks:DeleteCluster",
"eks:DeleteNodegroup",
"iam:ListPolicyVersions",
"iam:DeletePolicyVersion",
"iam:DeletePolicy",
"iam:DeleteRolePolicy",
"iam:DetachRolePolicy",
"iam:DeleteOpenIDConnectProvider",
"iam:DeleteRole",
"ecr:DeleteRepository"
],
"Resource": "*"
}
]
}
```

## Usage

Query the resources by providing the cloud provider name(`provider`) and the
Expand Down Expand Up @@ -68,16 +140,11 @@ The above command would list the resources that are older than 3 days.

In order to delete these resources, pass the `-delete` flag.

**NOTE:** Deleting resources is fully supported in Azure and GCP. Due to the
complexity of deleting the resources created in AWS, it's not implemented yet.
The test infrastructure for AWS involves a lot of individual components that
have to be managed independently, compared of Azure and GCP where resources
related to a cluster are related to one another and can be deleted all together.
If and when the complexity of the AWS test infrastructure is simplified,
deleting the resources can be easily implemented similar to the other providers.
Another issue that contributes to it is the stale resources that are reported
when listing resources via the Resource Groups Tagging API which makes it hard
to find out if a resource still exists or has been deleted without describing
the individual resource and checking their status.
**NOTE:** For AWS, unlike the other providers, a third party tool, [aws-nuke][aws-nuke],
is used. The `aws` provider may be removed in the future. It works in a very
limited manner using the Resource Groups Tagging API. The replacement,
`aws-nuke` provider, is capable of listing and deleting the resources properly.

Use the `-h` flag to list all the available options.

[aws-nuke]: https://github.com/ekristen/aws-nuke
31 changes: 21 additions & 10 deletions tools/reaper/go.mod
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
module github.com/fluxcd/test-infra/tools/reaper

go 1.20
go 1.22

replace github.com/fluxcd/test-infra/tftestenv => ../../tftestenv

require (
github.com/aws/aws-sdk-go v1.53.19
github.com/ekristen/aws-nuke/v3 v3.2.2
github.com/ekristen/libnuke v0.17.1
github.com/fluxcd/test-infra/tftestenv v0.0.0
github.com/k1LoW/duration v1.2.0
github.com/onsi/gomega v1.18.1
github.com/sirupsen/logrus v1.9.3
)

require (
Expand All @@ -22,6 +26,7 @@ require (
github.com/docker/docker-credential-helpers v0.6.4 // indirect
github.com/emicklei/go-restful v2.9.5+incompatible // indirect
github.com/evanphx/json-patch v4.12.0+incompatible // indirect
github.com/fatih/color v1.17.0 // indirect
github.com/go-logr/logr v1.2.0 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.19.5 // indirect
Expand All @@ -31,6 +36,8 @@ require (
github.com/google/gnostic v0.5.7-v3refs // indirect
github.com/google/go-containerregistry v0.11.0 // indirect
github.com/google/gofuzz v1.1.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gotidy/ptr v1.4.0 // indirect
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
Expand All @@ -39,35 +46,39 @@ require (
github.com/hashicorp/terraform-exec v0.18.1 // indirect
github.com/hashicorp/terraform-json v0.15.0 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.15.8 // indirect
github.com/mailru/easyjson v0.7.6 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mb0/glob v0.0.0-20160210091149-1eb79d2de6c4 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stevenle/topsort v0.2.0 // indirect
github.com/vbatts/tar-split v0.11.2 // indirect
github.com/zclconf/go-cty v1.13.0 // indirect
golang.org/x/crypto v0.5.0 // indirect
golang.org/x/mod v0.7.0 // indirect
golang.org/x/net v0.5.0 // indirect
golang.org/x/crypto v0.23.0 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/oauth2 v0.0.0-20220718184931-c8730f7fcb92 // indirect
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4 // indirect
golang.org/x/sys v0.4.0 // indirect
golang.org/x/term v0.4.0 // indirect
golang.org/x/text v0.6.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.20.0 // indirect
golang.org/x/term v0.20.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/api v0.24.1 // indirect
k8s.io/apimachinery v0.24.1 // indirect
k8s.io/client-go v0.24.1 // indirect
Expand Down
Loading

0 comments on commit c8e430d

Please sign in to comment.