Request for documentation on cloud-controller-manager component #4268
-
|
This is a follow on to my other ticket here: #4183 After playing around with the new rke2 version (1.27.1), I am noticing that the cluster automatically includes I am requesting documentation on how this component is recommended to be used with cluster-autoscaler in AWS. Since Kubernetes has moved away from in-tree providers, it is not obvious how one is expected to configure cloud-controller-manager in rke2 to be compatible with AWS. Am I expected to disable rke2's cloud-controller-manager and deploy the aws cloud provider? https://github.com/kubernetes/cloud-provider-aws Or is it possible to configure cluster-autoscaler to be compatible with the rke2 cloud-controller-manager in AWS? Simply setting |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
|
RKE2 uses the same stub cloud-controller-manager as K3s that just sets node internal/external IPs and clears the uninitialized taint. It is intended for use where there is no real cloud provider to integrate with, or where cloud provider integration is not desired. You can find docs on it here: https://docs.k3s.io/networking#deploying-an-external-cloud-controller-manager If you want to deploy the AWS cloud provider (https://github.com/kubernetes/cloud-provider-aws) you should start RKE2 with --disable-cloud-provider, and deploy cloud-provider-aws instead. |
Beta Was this translation helpful? Give feedback.
-
|
I thought that might be the case. Thank you for the clarification! |
Beta Was this translation helpful? Give feedback.
-
|
According to the doc https://docs.k3s.io/networking/networking-services#deploying-an-external-cloud-controller-manager It seems like if we deploy RKE2 on-prem, we still need to keep CCM. However, is it ok for us to set cloud-provider-name = "" instead and disable cloud controller ? This way we don't have the cloud controller at all and the nodes will not remain tainted. I believe lots of people are doing this way but not sure if this is ok? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @brandond it would be nice to have some clarification about the cloud-controller-manager component directly in the docs (maybe in https://docs.rke2.io/architecture). I'm new to RKE2 (coming from everything vanilla K8s) and it wasn't clear to me. |
Beta Was this translation helpful? Give feedback.
-
|
Leave it, unless you like having your nodes come up uninitialized. |
Beta Was this translation helpful? Give feedback.
-
|
I think you summed it up nicely in #4268 (comment). How about adding your answer to the RKE2 docs? |
Beta Was this translation helpful? Give feedback.
RKE2 uses the same stub cloud-controller-manager as K3s that just sets node internal/external IPs and clears the uninitialized taint. It is intended for use where there is no real cloud provider to integrate with, or where cloud provider integration is not desired. You can find docs on it here: https://docs.k3s.io/networking#deploying-an-external-cloud-controller-manager
If you want to deploy the AWS cloud provider (https://github.com/kubernetes/cloud-provider-aws) you should start RKE2 with --disable-cloud-provider, and deploy cloud-provider-aws instead.