-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding support for terraform private module source for git #7167
Changes from 4 commits
8c41565
d212a66
de5399d
ce877cc
d1a7ab7
3b29b9e
70a6bfe
188e3d8
e717d11
e2d4936
4b6b8b9
2149aad
f66fcc7
4db184f
ec388f7
89c32fa
d5cefb8
c3b89df
b843bc7
6023202
7f89d77
81e5598
2ada35f
fcdc510
8c178ca
c7091f0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
{"Resources":{"Applications.Core/applications@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":58},"Applications.Core/containers@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":123},"Applications.Core/environments@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":148},"Applications.Core/extenders@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":166},"Applications.Core/gateways@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":187},"Applications.Core/httpRoutes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":201},"Applications.Core/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":224},"Applications.Core/volumes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":261},"Applications.Dapr/pubSubBrokers@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":49},"Applications.Dapr/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":66},"Applications.Dapr/stateStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":84},"Applications.Datastores/mongoDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":50},"Applications.Datastores/redisCaches@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":69},"Applications.Datastores/sqlDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":88},"Applications.Messaging/rabbitMQQueues@2023-10-01-preview":{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":50}},"Functions":{"applications.core/extenders":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":262}]},"applications.core/secretstores":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":268}]},"applications.datastores/mongodatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":90}]},"applications.datastores/rediscaches":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":92}]},"applications.datastores/sqldatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":94}]},"applications.messaging/rabbitmqqueues":{"2023-10-01-preview":[{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":52}]}}} | ||
{"Resources":{"Applications.Core/applications@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":58},"Applications.Core/containers@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":123},"Applications.Core/environments@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":154},"Applications.Core/extenders@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":172},"Applications.Core/gateways@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":193},"Applications.Core/httpRoutes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":207},"Applications.Core/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":230},"Applications.Core/volumes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":267},"Applications.Dapr/pubSubBrokers@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":49},"Applications.Dapr/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":66},"Applications.Dapr/stateStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":84},"Applications.Datastores/mongoDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":50},"Applications.Datastores/redisCaches@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":69},"Applications.Datastores/sqlDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":88},"Applications.Messaging/rabbitMQQueues@2023-10-01-preview":{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":50}},"Functions":{"applications.core/extenders":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":268}]},"applications.core/secretstores":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":274}]},"applications.datastores/mongodatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":90}]},"applications.datastores/rediscaches":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":92}]},"applications.datastores/sqldatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":94}]},"applications.messaging/rabbitmqqueues":{"2023-10-01-preview":[{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":52}]}}} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,6 +18,7 @@ package v20231001preview | |
|
||
import ( | ||
"fmt" | ||
"reflect" | ||
"strings" | ||
|
||
v1 "github.com/radius-project/radius/pkg/armrpc/api/v1" | ||
|
@@ -63,6 +64,24 @@ func (src *EnvironmentResource) ConvertTo() (v1.DataModelInterface, error) { | |
} | ||
converted.Properties.Compute = *envCompute | ||
|
||
recipeConfig := src.Properties.RecipeConfig | ||
if recipeConfig != nil { | ||
if recipeConfig.Terraform != nil { | ||
if recipeConfig.Terraform.Authentication != nil { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can these conditions be collapsed? Or do we expect that there may be future logic that's unique to each config level? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In any case i think we still need to check nil at every level to avoid null pointer. Let me know if you think otherwise |
||
gitConfig := recipeConfig.Terraform.Authentication.Git | ||
if gitConfig != nil { | ||
pat := gitConfig.Pat | ||
p := map[string]datamodel.Secret{} | ||
for k, v := range pat { | ||
p[k] = datamodel.Secret{ | ||
SecretStore: to.String(v.SecretStore), | ||
} | ||
} | ||
converted.Properties.RecipeConfig.Terraform.Authentication.Git.PAT = p | ||
} | ||
} | ||
} | ||
} | ||
if src.Properties.Recipes != nil { | ||
envRecipes := make(map[string]map[string]datamodel.EnvironmentRecipeProperties) | ||
for resourceType, recipes := range src.Properties.Recipes { | ||
|
@@ -151,6 +170,27 @@ func (dst *EnvironmentResource) ConvertFrom(src v1.DataModelInterface) error { | |
dst.Properties.Recipes = recipes | ||
} | ||
|
||
if !reflect.DeepEqual(env.Properties.RecipeConfig, datamodel.RecipeConfigProperties{}) { | ||
recipeConfig := &RecipeConfigProperties{} | ||
if !reflect.DeepEqual(env.Properties.RecipeConfig.Terraform, datamodel.TerraformConfigProperties{}) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same as above, since this logic is specific to Terraform, maybe we can move it to a helper function There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can apply the same thing as I mentioned above but this time in reverse. This function -> RecipeConfig conversion function -> Terraform related conversion function In future, it would be very easy to add new types of provider conversions. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. created a separate func There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with the suggestion of creating a Terraform related conversion function. @vishwahiremat I know you have updated it to recipe config specific conversion, have you considered adding Terraform specific function to make it easy to extend the recipe config function in the future? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Vishwa and I discussed offline, we will abstract it out into Terraform specific function once we add support for other types. |
||
recipeConfig.Terraform = &TerraformConfigProperties{} | ||
if !reflect.DeepEqual(env.Properties.RecipeConfig.Terraform.Authentication, datamodel.AuthConfig{}) { | ||
recipeConfig.Terraform.Authentication = &AuthConfig{} | ||
if !reflect.DeepEqual(env.Properties.RecipeConfig.Terraform.Authentication.Git, datamodel.GitAuthConfig{}) { | ||
recipeConfig.Terraform.Authentication.Git = &GitAuthConfig{} | ||
if env.Properties.RecipeConfig.Terraform.Authentication.Git.PAT != nil { | ||
recipeConfig.Terraform.Authentication.Git.Pat = map[string]*Secret{} | ||
} | ||
} | ||
} | ||
} | ||
for k, v := range env.Properties.RecipeConfig.Terraform.Authentication.Git.PAT { | ||
recipeConfig.Terraform.Authentication.Git.Pat[k] = &Secret{ | ||
SecretStore: to.Ptr(v.SecretStore), | ||
} | ||
} | ||
dst.Properties.RecipeConfig = recipeConfig | ||
} | ||
if env.Properties.Providers != (datamodel.Providers{}) { | ||
dst.Properties.Providers = &Providers{} | ||
if env.Properties.Providers.Azure != (datamodel.ProvidersAzure{}) { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -117,6 +117,19 @@ func TestConvertVersionedToDataModel(t *testing.T) { | |
Scope: "/planes/aws/aws/accounts/140313373712/regions/us-west-2", | ||
}, | ||
}, | ||
RecipeConfig: datamodel.RecipeConfigProperties{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This needs a few different layers and we could run into nil pointer exception easily. We should test it thoroughly. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we remove the PAT map from this test, we can see if we run into nil pointer exception above in the original conversion logic. |
||
Terraform: datamodel.TerraformConfigProperties{ | ||
Authentication: datamodel.AuthConfig{ | ||
Git: datamodel.GitAuthConfig{ | ||
PAT: map[string]datamodel.Secret{ | ||
"dev.azure.com": datamodel.Secret{ | ||
SecretStore: "/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github", | ||
}, | ||
}, | ||
}, | ||
}, | ||
}, | ||
}, | ||
Recipes: map[string]map[string]datamodel.EnvironmentRecipeProperties{ | ||
ds_ctrl.MongoDatabasesResourceType: { | ||
"cosmos-recipe": datamodel.EnvironmentRecipeProperties{ | ||
|
@@ -368,6 +381,7 @@ func TestConvertDataModelToVersioned(t *testing.T) { | |
if tt.filename == "environmentresourcedatamodel.json" { | ||
require.Equal(t, "Azure/cosmosdb/azurerm", string(*versioned.Properties.Recipes[ds_ctrl.MongoDatabasesResourceType]["terraform-recipe"].GetRecipeProperties().TemplatePath)) | ||
require.Equal(t, recipes.TemplateKindTerraform, string(*versioned.Properties.Recipes[ds_ctrl.MongoDatabasesResourceType]["terraform-recipe"].GetRecipeProperties().TemplateKind)) | ||
require.Equal(t, "/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github", string(*versioned.Properties.RecipeConfig.Terraform.Authentication.Git.Pat["dev.azure.com"].SecretStore)) | ||
switch c := recipeDetails.(type) { | ||
case *TerraformRecipeProperties: | ||
require.Equal(t, "1.1.0", string(*c.TemplateVersion)) | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,6 +16,19 @@ | |
"scope": "/planes/aws/aws/accounts/140313373712/regions/us-west-2" | ||
} | ||
}, | ||
"recipeConfig": { | ||
"terraform": { | ||
"authentication": { | ||
"git": { | ||
"pat": { | ||
"dev.azure.com":{ | ||
"secretStore":"/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github" | ||
} | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What if this part doesn't exist (by mistake of the operator), would this cause a nil pointer exception? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I dont think it will nil pointer if pat property doesnt exist There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add a unit test for this? |
||
} | ||
} | ||
} | ||
}, | ||
"recipes": { | ||
"Applications.Datastores/mongoDatabases":{ | ||
"cosmos-recipe": { | ||
|
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since these lines are specific to Terraform, maybe we should move this logic to a helper function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with @sk593. We can have a function that takes care of the recipeConfig conversion and that function can make a call to another function which takes care of terraform related stuff conversion. With this approach we could keep things simple. It would be more readable, easier to debug and write unit tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done