Allow operators to specify Helm charts as part of their Radius environment

[jkotalik]

Overview

With Radius environments starting to become more feature rich, we would like to be able to describe infrastructure components as part of the environment as well. For example:

• Requiring Dapr for an environment
• Requiring OSM for traffic splitting
• Requiring a specific API gateway for routing

The common way to install these pieces is through a Helm chart. As part of rad env init/rad install, we already use the helm client to install dapr and contour. Being able to specify it outside of rad env init would be useful in setting up a cluster.

We are motivated to remove default pieces like dapr and contour as part of default installations in the future for Radius (#2952). This will improve the footprint of Radius on a Kubernetes cluster and allow people to specify which exact version of these dependencies to use, rather than the one baked into Radius.

To enable this, being able to specify helm resources in bicep would be a great stepping-stone for this functionality.

User scenarios

As an infrastructure engineer, I would like to specify my companies helm chart implementation of MongoDB to ensure that all engineers are using the compliant version of MongoDB.

As a Dapr developer, I'd like to control the version of dapr installed in the cluster rather than having Radius install it for me.

As an infrastructure engineer, I'd like to enforce specific versions for different environment components based on the required version

Proposal

There are two main pieces that need to be added here:

• Adding support for specifying helm charts in bicep
• Adding information to the environment to require specific versions for infrastructure pieces

Helm chart support in bicep

To start with, we should add support for helm releases, which allows for charts to be installed on a cluster.

import helm as helm {
  kubeConfig: ... // if required to specify credentials to connect
}

resource dapr 'helm/release@v1' {
  properties: {
    name: 'my-dapr-release' // name of the release in the cluster, TODO can we enforce uniqueness here? What happens when there is a conflict, required
    namespace: 'radius-system' // TODO determine defaulting behavior if we don't require this.
    repository: 'https://dapr.github.io/helm-charts/' // Required
    chart: 'dapr' // Required
    version: '1.8.4'
    
    sets: [ // To override values
      {
        name: 'global.tag'
        value: '1.8.3'
       }
    ]
  }
}

This is effectively a minimal set of properties for a helm chart. We can add more as we go (wait, repository certificate authorization, timeout, etc.). For inspiration, see https://github.com/hashicorp/terraform-provider-helm/blob/main/helm/resource_release.go.

Open questions:

• A big use case that will be missing is the helm chart not being able to be specified locally. How bad is that? Could we make this easier for people to upload their helm charts or provide documentation to make this easier?
• Values in sets, should they be treated as secrets? Should we try to treat everything in sets as a secret?

Tasks

• Add swagger definitions for helm chart, similar to how we created definitions for Kubernetes and Radius
• Add bicep type converter support for Kubernetes (see https://github.com/project-radius/radius/tree/main/hack/bicep-types-radius for more details)
• Add bicep changes to load types for helm: https://github.com/project-radius/bicep/tree/bicep-extensibility/src/Bicep.Core/TypeSystem/K8s
• Add unit tests in bicep to confirm helm extensibility types can be compiled
• Confirm in VSCode that typing full helm release type and import gives great completion

Helm chart support in the Deployment Engine

To support the helm extensibility provider, it will be a simple change to add the route to the helm provider in the deployment engine and make sure request are routed to it for the import.

Tasks

• Add Helm Bicep extensibility route to deployment engine

Helm chart support as a bicep extensibility provider

The helm chart bicep extensibility provider will need to be written in golang due to Helm SDK's lack of support for C#. This will require using the extensibility swagger defined here to implement the extensibility API contract in golang.

From there, it will be a fairly straightforward invocation of the helm client APIs. We already do that today for many client-side things: https://github.com/project-radius/radius/blob/main/pkg/cli/helm/contourclient.go.

Tasks

• Implement skeleton API for Get, Save, PreviewSave, and Delete
• Implement models for extensibility types, making sure they are json round trippable with unit tests
• Implement mocks for helm installations/releases
• Implement Save with tests -> install
• Implement Get with tests -> get
• Implement Delete with tests -> uninstall
• Implement PreviewSave -> noop
• Add integration test with Radius, installing dapr as part of a helm chart

AB#3979

[rynowak]

One open question I'd like to add - do we want to go the Bicep extensibility route for this? Or do we want to integrate this with UCP and keep the Bicep integration minimal.

• I think the upside of UCP is that it ends up being applicable to a lot more scenarios.
• I think the upside of Bicep extensibility is a clear path to be useful separate from UCP/Radius.

[jkotalik]

We are going to discuss with the ARM team as well, there was interest in ownership of the helm provider by them.