Improve deployment of remote cluster resources

[nrfox]

What do you want to improve?

When you are deploying Kiali to read from a remote cluster, you need to deploy some resources to give it access. The way that you do that today is by using the kiali-prepare-remote-cluster.sh script which in turn calls helm template for those resources.

This works well for getting started but it would be nice to have a way of tracking these resources outside of this script either within the helm chart or within the Kiali CR. A concrete example of this is when the RBAC roles in those templates change across versions. Users would need to re-run the script to regenerate the templates. Instead if there was a helm template or the Kiali CR tracked these resources, they would get automatically updated when you update your version of Kiali in the remote cluster.

What is the new behavior?

Provide a way of deploying/tracking remote resources outside of the hack script. Could be either a separate helm chart/CR or an additional remoteDeployment option on the existing chart/CR to only deploy the remote templates.

[jmazzitelli]

After thinking about this, I think it would be best to do this with a helm chart. We normally wouldn't want users to install the operator in their remote cluster if all they need is those remote resources installed. The operator is really to manage a full running Kiali server instance... the helm chart seems more natural to use to simply deploy some resources.

I'll think about this some more....maybe putting it in the operator will help the bulk of users, OpenShift as well as non-OS

[jmazzitelli]

@nrfox the hack script creates resources in both clusters (the kiali cluster and the remote cluster). Is this enhancement just for the operator in the remote cluster to create the remote cluster resources (basically, the SA and its secret, the clusterRole, and the clusterRoleBinding)??

How does the user then know how to create the remote cluster secret on the Kiali cluster side (and also to configure Kiali CR for it - this is the spec.clustering.clusters.secret_name setting)?? They could create the secret with the correct label so auto-discovery would work and no change needed in the Kiali CR, but the user would need to know to do that, too. My guess is - the user needs to RTFM. (?)

[nrfox]

@nrfox the hack script creates resources in both clusters (the kiali cluster and the remote cluster). Is this enhancement just for the operator in the remote cluster to create the remote cluster resources (basically, the SA and its secret, the clusterRole, and the clusterRoleBinding)??

Yes all those except the secret/token. I think we'll want to give users more flexibility on how they manage this and the tokens could potentially be more ephemeral whereas the other resources will not be. This adds a step to the Kiali multi-cluster setup e.g. kubectl create token <remote-service-account> --context remote-cluster` but I think the added flexibility is worth it.

How does the user then know how to create the remote cluster secret on the Kiali cluster side (and also to configure Kiali CR for it - this is the spec.clustering.clusters.secret_name setting)?? They could create the secret with the correct label so auto-discovery would work and no change needed in the Kiali CR, but the user would need to know to do that, too. My guess is - the user needs to RTFM. (?)

Yes you would still need to create the remote cluster secret on the Kiali cluster side. There's no way, that I know of, to automate this step without having some higher level automation that already has access to both clusters. In this case the thing that has access to both clusters is the human cluster admin.

[jmazzitelli]

@nrfox For remote clusters - do we expect or is it required that the Kiali roles created are ClusterRoles giving cluster-wide access? Or do we provide the option to provide a list of discovery selectors in the CR and the operator creates Roles for only those namespaces?

Iin our current hack script, it always creates cluster_wide_access=true roles (meaning a ClusterRole/Binding is created giving access to the full cluster). Is that how we want it?

[nrfox]

@nrfox For remote clusters - do we expect or is it required that the Kiali roles created are ClusterRoles giving cluster-wide access? Or do we provide the option to provide a list of discovery selectors in the CR and the operator creates Roles for only those namespaces?

Previously there was nothing to reconcile those roles because the operator on the Kiali side cluster couldn't create the roles on the remote cluster but if there's now a Kiali operator on the remote cluster then it could reconcile them.

[jmazzitelli]

We should probably add this to the kiali.io docs. I think we add something to discuss using this new operator feature to deploy the remote resources ... something here https://kiali.io/docs/configuration/multi-cluster/

UPDATE: this is done: kiali/kiali.io#831

[jmazzitelli]

PRs are all done and ready for additional review and testing. My testing shows everything is good so far.

• Server PR: multi-cluster setup - no need for script to show only specific templates - the helm chart has a flag for this now #7883
• Operator PR: provide a way to tell the operator to create only the remote cluster resources kiali-operator#836
• Helm chart PR: support remote_cluster_resources_only=true and support redirect_uris for openshift oauthclient helm-charts#295
• Docs PR: talk about using the operator to create the remote cluster resources kiali.io#831