Skip to content

Commit

Permalink
Adding support for terraform private module source for git (radius-pr…
Browse files Browse the repository at this point in the history 
…oject#7167)

- Added a RecipeConfig property to the environment
  - Typesspec changes
  - Conversion and unit tests
  - Datamodel changes
- Adding changes to the config to append the template path with
credential informaiton
   - changes to query list secret to get secret information
   - creating credential appended template path

Design Doc: radius-project/design-notes#37

<!--

Please select **one** of the following options that describes your
change and delete the others. Clearly identifying the type of change you
are making will help us review your PR faster, and is used in authoring
release notes.

If you are making a bug fix or functionality change to Radius and do not
have an associated issue link please create one now.

-->

- This pull request adds or changes features of Radius and has an
approved issue (issue link required).
- This pull request is a minor refactor, code cleanup, test improvement,
or other maintenance task and doesn't change the functionality of Radius
(issue link optional).

<!--

Please update the following to link the associated issue. This is
required for some kinds of changes (see above).

-->

Fixes: radius-project#6911

---------

Signed-off-by: Vishwanath Hiremath <[email protected]>
  • Loading branch information
@vishwahiremat @lakshmimsft
vishwahiremat authored and lakshmimsft committed Feb 27, 2024
1 parent 6885574 commit e3cbe2d
Show file tree
Hide file tree
Showing 38 changed files with 1,459 additions and 37 deletions.
2 changes: 1 addition & 1 deletion ...bicep-types-radius/generated/applications/applications.core/2023-10-01-preview/types.json
View file

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion hack/bicep-types-radius/generated/index.json
View file
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"Resources":{"Applications.Core/applications@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":58},"Applications.Core/containers@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":123},"Applications.Core/environments@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":148},"Applications.Core/extenders@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":166},"Applications.Core/gateways@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":187},"Applications.Core/httpRoutes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":201},"Applications.Core/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":224},"Applications.Core/volumes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":261},"Applications.Dapr/pubSubBrokers@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":49},"Applications.Dapr/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":66},"Applications.Dapr/stateStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":84},"Applications.Datastores/mongoDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":50},"Applications.Datastores/redisCaches@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":69},"Applications.Datastores/sqlDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":88},"Applications.Messaging/rabbitMQQueues@2023-10-01-preview":{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":50}},"Functions":{"applications.core/extenders":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":262}]},"applications.core/secretstores":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":268}]},"applications.datastores/mongodatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":90}]},"applications.datastores/rediscaches":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":92}]},"applications.datastores/sqldatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":94}]},"applications.messaging/rabbitmqqueues":{"2023-10-01-preview":[{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":52}]}}}
{"Resources":{"Applications.Core/applications@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":58},"Applications.Core/containers@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":123},"Applications.Core/environments@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":154},"Applications.Core/extenders@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":172},"Applications.Core/gateways@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":193},"Applications.Core/httpRoutes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":207},"Applications.Core/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":230},"Applications.Core/volumes@2023-10-01-preview":{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":267},"Applications.Dapr/pubSubBrokers@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":49},"Applications.Dapr/secretStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":66},"Applications.Dapr/stateStores@2023-10-01-preview":{"RelativePath":"applications/applications.dapr/2023-10-01-preview/types.json","Index":84},"Applications.Datastores/mongoDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":50},"Applications.Datastores/redisCaches@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":69},"Applications.Datastores/sqlDatabases@2023-10-01-preview":{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":88},"Applications.Messaging/rabbitMQQueues@2023-10-01-preview":{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":50}},"Functions":{"applications.core/extenders":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":268}]},"applications.core/secretstores":{"2023-10-01-preview":[{"RelativePath":"applications/applications.core/2023-10-01-preview/types.json","Index":274}]},"applications.datastores/mongodatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":90}]},"applications.datastores/rediscaches":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":92}]},"applications.datastores/sqldatabases":{"2023-10-01-preview":[{"RelativePath":"applications/applications.datastores/2023-10-01-preview/types.json","Index":94}]},"applications.messaging/rabbitmqqueues":{"2023-10-01-preview":[{"RelativePath":"applications/applications.messaging/2023-10-01-preview/types.json","Index":52}]}}}
55 changes: 55 additions & 0 deletions pkg/corerp/api/v20231001preview/environment_conversion.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package v20231001preview

import (
"fmt"
"reflect"
"strings"

v1 "github.com/radius-project/radius/pkg/armrpc/api/v1"
Expand Down Expand Up @@ -62,6 +63,7 @@ func (src *EnvironmentResource) ConvertTo() (v1.DataModelInterface, error) {
return nil, err
}
converted.Properties.Compute = *envCompute
converted.Properties.RecipeConfig = toRecipeConfigDatamodel(src.Properties.RecipeConfig)

if src.Properties.Recipes != nil {
envRecipes := make(map[string]map[string]datamodel.EnvironmentRecipeProperties)
Expand Down Expand Up @@ -150,6 +152,7 @@ func (dst *EnvironmentResource) ConvertFrom(src v1.DataModelInterface) error {
}
dst.Properties.Recipes = recipes
}
dst.Properties.RecipeConfig = fromRecipeConfigDatamodel(env.Properties.RecipeConfig)

if env.Properties.Providers != (datamodel.Providers{}) {
dst.Properties.Providers = &Providers{}
Expand Down Expand Up @@ -180,6 +183,58 @@ func (dst *EnvironmentResource) ConvertFrom(src v1.DataModelInterface) error {
return nil
}

func toRecipeConfigDatamodel(config *RecipeConfigProperties) datamodel.RecipeConfigProperties {
if config != nil {
recipeConfig := datamodel.RecipeConfigProperties{}
if config.Terraform != nil {
recipeConfig.Terraform = datamodel.TerraformConfigProperties{}
if config.Terraform.Authentication != nil {
recipeConfig.Terraform.Authentication = datamodel.AuthConfig{}
gitConfig := config.Terraform.Authentication.Git
if gitConfig != nil {
recipeConfig.Terraform.Authentication.Git = datamodel.GitAuthConfig{}
if gitConfig.Pat != nil {
p := map[string]datamodel.SecretConfig{}
for k, v := range gitConfig.Pat {
p[k] = datamodel.SecretConfig{
Secret: to.String(v.Secret),
}
}
recipeConfig.Terraform.Authentication.Git.PAT = p
}
}
}
}
return recipeConfig
}
return datamodel.RecipeConfigProperties{}
}

func fromRecipeConfigDatamodel(config datamodel.RecipeConfigProperties) *RecipeConfigProperties {
if !reflect.DeepEqual(config, datamodel.RecipeConfigProperties{}) {
recipeConfig := &RecipeConfigProperties{}
if !reflect.DeepEqual(config.Terraform, datamodel.TerraformConfigProperties{}) {
recipeConfig.Terraform = &TerraformConfigProperties{}
if !reflect.DeepEqual(config.Terraform.Authentication, datamodel.AuthConfig{}) {
recipeConfig.Terraform.Authentication = &AuthConfig{}
if !reflect.DeepEqual(config.Terraform.Authentication.Git, datamodel.GitAuthConfig{}) {
recipeConfig.Terraform.Authentication.Git = &GitAuthConfig{}
if config.Terraform.Authentication.Git.PAT != nil {
recipeConfig.Terraform.Authentication.Git.Pat = map[string]*SecretConfig{}
for k, v := range config.Terraform.Authentication.Git.PAT {
recipeConfig.Terraform.Authentication.Git.Pat[k] = &SecretConfig{
Secret: to.Ptr(v.Secret),
}
}
}
}
}
}
return recipeConfig
}
return nil
}

func toEnvironmentComputeDataModel(h EnvironmentComputeClassification) (*rpv1.EnvironmentCompute, error) {
switch v := h.(type) {
case *KubernetesCompute:
Expand Down
21 changes: 21 additions & 0 deletions pkg/corerp/api/v20231001preview/environment_conversion_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,13 @@ func TestConvertVersionedToDataModel(t *testing.T) {
Scope: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testGroup",
},
},
RecipeConfig: datamodel.RecipeConfigProperties{
Terraform: datamodel.TerraformConfigProperties{
Authentication: datamodel.AuthConfig{
Git: datamodel.GitAuthConfig{},
},
},
},
Recipes: map[string]map[string]datamodel.EnvironmentRecipeProperties{
ds_ctrl.MongoDatabasesResourceType: {
"cosmos-recipe": datamodel.EnvironmentRecipeProperties{
Expand Down Expand Up @@ -117,6 +124,19 @@ func TestConvertVersionedToDataModel(t *testing.T) {
Scope: "/planes/aws/aws/accounts/140313373712/regions/us-west-2",
},
},
RecipeConfig: datamodel.RecipeConfigProperties{
Terraform: datamodel.TerraformConfigProperties{
Authentication: datamodel.AuthConfig{
Git: datamodel.GitAuthConfig{
PAT: map[string]datamodel.SecretConfig{
"dev.azure.com": {
Secret: "/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github",
},
},
},
},
},
},
Recipes: map[string]map[string]datamodel.EnvironmentRecipeProperties{
ds_ctrl.MongoDatabasesResourceType: {
"cosmos-recipe": datamodel.EnvironmentRecipeProperties{
Expand Down Expand Up @@ -368,6 +388,7 @@ func TestConvertDataModelToVersioned(t *testing.T) {
if tt.filename == "environmentresourcedatamodel.json" {
require.Equal(t, "Azure/cosmosdb/azurerm", string(*versioned.Properties.Recipes[ds_ctrl.MongoDatabasesResourceType]["terraform-recipe"].GetRecipeProperties().TemplatePath))
require.Equal(t, recipes.TemplateKindTerraform, string(*versioned.Properties.Recipes[ds_ctrl.MongoDatabasesResourceType]["terraform-recipe"].GetRecipeProperties().TemplateKind))
require.Equal(t, "/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github", string(*versioned.Properties.RecipeConfig.Terraform.Authentication.Git.Pat["dev.azure.com"].Secret))
switch c := recipeDetails.(type) {
case *TerraformRecipeProperties:
require.Equal(t, "1.1.0", string(*c.TemplateVersion))
Expand Down
7 changes: 7 additions & 0 deletions pkg/corerp/api/v20231001preview/testdata/environmentresource-with-workload-identity.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@
"scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testGroup"
}
},
"recipeConfig": {
"terraform": {
"authentication": {
"git": {}
}
}
},
"recipes": {
"Applications.Datastores/mongoDatabases":{
"cosmos-recipe": {
Expand Down
13 changes: 13 additions & 0 deletions pkg/corerp/api/v20231001preview/testdata/environmentresource.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,19 @@
"scope": "/planes/aws/aws/accounts/140313373712/regions/us-west-2"
}
},
"recipeConfig": {
"terraform": {
"authentication": {
"git": {
"pat": {
"dev.azure.com":{
"secret":"/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github"
}
}
}
}
}
},
"recipes": {
"Applications.Datastores/mongoDatabases":{
"cosmos-recipe": {
Expand Down
7 changes: 7 additions & 0 deletions ...rp/api/v20231001preview/testdata/environmentresourcedatamodel-with-workload-identity.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@
"scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/testGroup"
}
},
"recipeConfig": {
"terraform": {
"authentication": {
"git": {}
}
}
},
"recipes": {
"Applications.Datastores/mongoDatabases":{
"cosmos-recipe": {
Expand Down
13 changes: 13 additions & 0 deletions pkg/corerp/api/v20231001preview/testdata/environmentresourcedatamodel.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,19 @@
"scope": "/planes/aws/aws/accounts/140313373712/regions/us-west-2"
}
},
"recipeConfig": {
"terraform": {
"authentication": {
"git": {
"pat": {
"dev.azure.com":{
"secret":"/planes/radius/local/resourcegroups/default/providers/Applications.Core/secretStores/github"
}
}
}
}
}
},
"recipes": {
"Applications.Datastores/mongoDatabases":{
"cosmos-recipe": {
Expand Down
39 changes: 39 additions & 0 deletions pkg/corerp/api/v20231001preview/zz_generated_models.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit e3cbe2d

Please sign in to comment.