Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testenv.Destroy for deleting resources #29

Merged
merged 2 commits into from
Jan 8, 2024
Merged

Conversation

darkowlzz
Copy link
Contributor

@darkowlzz darkowlzz commented Jan 5, 2024

testenv.Destroy() can be used to delete resources from a terraform configuration directory without going through the whole infrastructure set up steps. This is to be used in CI to always run at the end to make sure that everything has been destroyed after the test run.

In ideal case, this won't be needed, as the environment constructor, testenv.New(), has signal handlers to gracefully delete the infrastructure. But if the whole test process gets terminated, a new process can just run testenv.Destroy() to perform the cleanup separately, ensuring everything gets deleted.

This is based on a recently failure in pkg repo OCI GCP integration test. The GKE cluster creation took more than 30 minutes. The test runner timed out and terminated with the following error:

*** Test killed with quit: ran too long (31m0s).

In the GKE dashboard, it showed that some of the provisioned nodes were not healthy. This happened for the first time.
Refer: https://github.com/fluxcd/pkg/actions/runs/7409263553/job/20159178143#step:13:338

To handle such scenarios, a new step can be added in github workflows to run a new process that just runs terraform destroy with all the configurations that were used to provision the infrastructure. This can run with its own timeout, without affecting the test run.

In addition, the testenv Environment now has tfApplyOptions and tfDestroyOptions to configure the terraform apply and destroy options. This is added due to the need to ignore lock state in destroy-only mode.

See fluxcd/pkg#712 for an example of how this is used.

@darkowlzz darkowlzz added the area/tftestenv Tftestenv package related issues and pull requests label Jan 5, 2024
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

testenv.Destroy() can be used to delete resources from a terraform
configuration directory without going through the whole infrastructure
set up steps. This is to be used in CI to always run at the end to make
sure that everything has been destroyed after the test run.

In ideal case, this won't be needed, as the environment constructor,
testenv.New(), has signal handlers to gracefully delete the
infrastructure. But if the whole test process gets terminated, a new
process can just run testenv.Destroy() to perform the cleanup.

Signed-off-by: Sunny <[email protected]>
@darkowlzz darkowlzz force-pushed the testenv-destroy branch 2 times, most recently from df0b7e4 to 84fb729 Compare January 8, 2024 13:24
The terraform apply and destroy operations can now be configured with
testenv EnvironmentOption. These can be used to configure the behavior
of apply and destroy.

Signed-off-by: Sunny <[email protected]>
@darkowlzz darkowlzz merged commit b58e0c4 into main Jan 8, 2024
2 checks passed
@darkowlzz darkowlzz deleted the testenv-destroy branch January 8, 2024 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tftestenv Tftestenv package related issues and pull requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants