Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Implementing Terraform driver #5533

Closed
wants to merge 10 commits into from
Closed

Conversation

kachawla
Copy link
Contributor

@kachawla kachawla commented May 6, 2023

Description

Initial prototype that can enable us to catch any unknown blockers early on. Doesn't include unit tests and optimizations yet.

Issue reference

Fixes: #issue_number

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Adds necessary unit tests for change
  • Adds necessary E2E tests for change
  • Unit tests passing
  • Extended the documentation / Created issue for it

@kachawla kachawla requested a review from rynowak May 6, 2023 01:58
@github-actions
Copy link

github-actions bot commented May 6, 2023

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 6, 2023

Test Results

2 541 tests  +3   2 534 ✔️ +3   1m 55s ⏱️ +2s
   230 suites ±0          7 💤 ±0 
       1 files   ±0          0 ±0 

Results for commit ecec4a9. ± Comparison against base commit 473dbdf.

This pull request removes 2 and adds 5 tests. Note that renamed tests count towards both.
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/79b6c8f4-7c86-44d8-b31b-bb3f290e77b5
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/79b6c8f4-7c86-44d8-b31b-bb3f290e77b5#01
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/f61e7698-ed53-499b-87b9-b9870506385f
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/f61e7698-ed53-499b-87b9-b9870506385f#01
github.com/project-radius/radius/pkg/ucp/resources ‑ Test_ParseProviderScope
github.com/project-radius/radius/pkg/ucp/resources ‑ Test_ParseProviderScope/AWS_provider_region
github.com/project-radius/radius/pkg/ucp/resources ‑ Test_ParseProviderScope/Azure_provider_resource_group

♻️ This comment has been updated with latest results.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from 065f952 to 432d6bf Compare May 8, 2023 19:36
@github-actions
Copy link

github-actions bot commented May 8, 2023

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from 432d6bf to b052369 Compare May 8, 2023 22:22
@github-actions
Copy link

github-actions bot commented May 8, 2023

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from b052369 to abdfb9e Compare May 8, 2023 22:46
@github-actions
Copy link

github-actions bot commented May 8, 2023

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from abdfb9e to 161b71e Compare May 8, 2023 23:14
@github-actions
Copy link

github-actions bot commented May 8, 2023

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 8, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 161b71e
Unique ID e5a81fe65f
Image tag pr-e5a81fe65f
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-e5a81fe65f
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-e5a81fe65f

Test Status

✅ Building Radius and pushing container images for functional test...
✅ Test image build succeeded
✅ Starting samples functional tests...
✅ Starting corerp functional tests...
✅ Starting ucp functional tests...
❌ Failed to install Radius for samples functional test. Please check the logs for more details.
❌ samples functional test failed. Please check the logs for more details.

@@ -363,7 +363,8 @@
## EnvironmentRecipeProperties
### Properties
* **parameters**: any: Any object
* **templatePath**: string (Required): Path to the template provided by the recipe. Currently only link to Azure Container Registry is supported.
* **templateKind**: string: Format of the template provided by the recipe. Currenlty only Bicep and Terraform templates are supported.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to separate the app model change from the other changes so we can get it in sooner?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a quick solution for me to be able to integrate end to end. I expect it to be added separately as a required field (which will result into updating all bicep templates as well) - have a separate task tracking that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went ahead and separated it out here #5575, since it will be good to get users onboarded to this change sooner than later as you mentioned as well, so targeting to include it in the release next week.

@@ -63,12 +63,16 @@ spec:
volumeMounts:
- name: config-volume
mountPath: /etc/config
- name: terraform
mountPath: /terraform
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something to consider here - how does the RP know to use this directory? Is this configurable?


const (
installDirRoot = "/terraform/install"
workingDirRoot = "/terraform/exec"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we find a way to make this configurable? This will cause problems if these paths are used in dev scenarios.

// NewTerraformDriver creates a new instance of driver to execute a Terraform recipe.
func NewTerraformDriver() Driver {
return &terraformDriver{}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: There's no need to define a "constructor" unless there's some initialization to do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet, but we will likely initialize things needed to setup creds.

}

type TerraformDefinition struct {
RequiredProviders map[string]interface{} `json:"required_providers"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use camelCase for json property names

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is driven by Terraform's syntax so we don't really have much of a control in what it should look like. I'll test if Terraform accepts camel casing, haven't seen that so far.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked if Terraform accepts camelCase, it does not. Let me know if I misunderstood your comment.

The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
╷
│ Error: Extraneous JSON object property
│ 
│   on main.tf.json line 3, in terraform:
│    3:     "requiredProviders": {
│ 
│ No argument or block type is named "requiredProviders". Did you mean "required_providers"?
╵

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AH I get it. If this is an existing format we should obviously follow that 😆

logger.Info(fmt.Sprintf("Deploying recipe: %q, template: %q", recipe.Name, definition.TemplatePath))

// Create Terraform installation directory
installDir := filepath.Join(installDirRoot, util.NormalizeStringToLower(recipe.ResourceID), uuid.NewString())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to create a unique install directory for each execution of the driver? Can you leave a comment explaining why instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is emphemeral and based on a UUID - why do we need to use the resource ID at all? Is a UUID not sufficiently unique?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had added a comment regarding this below before Install is called - I don't know yet if we need unique installation per request or we should reuse the installation. We can optimize this as we learn more.

// We should look into checking if an exsiting installation of Terraform is available.
// For initial iteration we will always install Terraform. Optimizations can be made later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is emphemeral and based on a UUID - why do we need to use the resource ID at all? Is a UUID not sufficiently unique?

Not for uniqueness, but for tracking and debugging if we ever need to figure out which resource the directory was created for.

@github-actions
Copy link

github-actions bot commented May 8, 2023

62.0

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.0 %
  • main branch coverage: 62.2 %
  • diff coverage: -.2 %

The coverage result does not include the functional test coverage.

Version string `json:"version"`
}

func (d *terraformDriver) Execute(ctx context.Context, configuration recipes.Configuration, recipe recipes.Metadata, definition recipes.Definition) (recipeOutput *recipes.RecipeOutput, err error) {
Copy link
Contributor

@rynowak rynowak May 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not too early to start thinking about testability. There's a lot of code in this function that does IO and can't easily be compartmentalized for unit testing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example you could pull the installer into an interface or separate function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is first pass at implementation to get us to a state where we can quickly figure out if there are any unknown unknowns we haven't come across yet. So I haven't really spent time on optimizing for testability, polishing up the code, and making values configurable (that you mentioned in other comments).

@github-actions
Copy link

github-actions bot commented May 9, 2023

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 9, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 1abf860
Unique ID e59e3d6fbb
Image tag pr-e59e3d6fbb
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-e59e3d6fbb
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-e59e3d6fbb

Test Status

✅ Building Radius and pushing container images for functional test...
✅ Test image build succeeded
✅ Starting corerp functional tests...
✅ Starting ucp functional tests...
✅ Starting samples functional tests...
❌ Failed to install Radius for corerp functional test. Please check the logs for more details.
❌ corerp functional test failed. Please check the logs for more details.
❌ Failed to install Radius for ucp functional test. Please check the logs for more details.
❌ ucp functional test failed. Please check the logs for more details.
❌ Failed to install Radius for samples functional test. Please check the logs for more details.
❌ samples functional test failed. Please check the logs for more details.

@github-actions
Copy link

github-actions bot commented May 9, 2023

62.0

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.0 %
  • main branch coverage: 62.2 %
  • diff coverage: -.2 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from 1abf860 to 0aec081 Compare May 11, 2023 16:36
@github-actions
Copy link

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions

This comment has been minimized.

@github-actions
Copy link

github-actions bot commented May 11, 2023

⌛ Building Radius and pushing container images for functional tests...

✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting samples functional tests...
⌛ Starting ucp functional tests...
⌛ Starting corerp functional tests...
❌ Failed to install Radius for samples functional test. Please check the logs for more details
❌ samples functional test failed. Please check the logs for more details
❌ Failed to install Radius for ucp functional test. Please check the logs for more details

@kachawla kachawla force-pushed the kachawla/terraform-init branch from 0aec081 to fbab6b2 Compare May 11, 2023 19:52
@github-actions
Copy link

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 11, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref fbab6b2
Unique ID 695c6b9452
Image tag pr-695c6b9452
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-695c6b9452
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-695c6b9452
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-695c6b9452

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp functional tests...
⌛ Starting ucp functional tests...
⌛ Starting samples functional tests...
❌ Failed to install Radius for corerp functional test. Please check the logs for more details
❌ corerp functional test failed. Please check the logs for more details

@kachawla kachawla force-pushed the kachawla/terraform-init branch from fbab6b2 to 7e1042e Compare May 11, 2023 20:06
@github-actions
Copy link

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 11, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 7e1042e
Unique ID b5d0485ac0
Image tag pr-b5d0485ac0
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-b5d0485ac0
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-b5d0485ac0
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-b5d0485ac0

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting samples functional tests...
⌛ Starting corerp functional tests...
✅ samples functional tests succeeded
⌛ Starting ucp functional tests...
✅ ucp functional tests succeeded
✅ corerp functional tests succeeded

@github-actions
Copy link

62.2

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.2 %
  • main branch coverage: 62.5 %
  • diff coverage: -.3 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from 7e1042e to 0b73442 Compare May 12, 2023 05:52
@github-actions
Copy link

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 12, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 0b73442
Unique ID 8f2d1f3c6e
Image tag pr-8f2d1f3c6e
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-8f2d1f3c6e
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-8f2d1f3c6e
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-8f2d1f3c6e

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting samples functional tests...
⌛ Starting corerp functional tests...
⌛ Starting ucp functional tests...
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ corerp functional tests succeeded

@github-actions
Copy link

62.0

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.0 %
  • main branch coverage: 62.5 %
  • diff coverage: -.5 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/terraform-init branch from 0b73442 to ecec4a9 Compare May 12, 2023 06:11
@github-actions
Copy link

You are editing files which require a docs update. Please ensure you've made the appropriate changes to the docs and submitted a PR.

  • Swagger changes require reference edge docs to be updated here
  • Make sure to create a branch and submit a PR into the edge branch instead of the default latest branch, as the swagger changes will not be available until the next release.

For more information on contributing to docs please visit https://docs.radapp.dev/contributing/docs/.

@github-actions
Copy link

github-actions bot commented May 12, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref ecec4a9
Unique ID c229cba486
Image tag pr-c229cba486
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-c229cba486
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-c229cba486
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-c229cba486

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp functional tests...
⌛ Starting corerp functional tests...
⌛ Starting samples functional tests...
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ corerp functional tests succeeded

@github-actions
Copy link

62.0

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.0 %
  • main branch coverage: 62.5 %
  • diff coverage: -.5 %

The coverage result does not include the functional test coverage.


func (d *terraformDriver) fetchAzureCredentials() (*credentials.AzureCredential, error) {
secretProvider := provider.NewSecretProvider(provider.SecretProviderOptions{
Provider: provider.TypeKubernetesSecret,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provider: provider.TypeKubernetesSecret -- this needs to be fixed to use whatever provider UCP is initialized to use.

desc string
scope string
expectedScopeSegments []ScopeSegment
}{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any bugs or just realize we were missing test coverage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just missing test coverage, making sure Parse continues to work for scope now that Terraform implementation uses it :)

@kachawla kachawla closed this May 26, 2023
@kachawla kachawla deleted the kachawla/terraform-init branch February 24, 2024 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants