feat: each node pool can now have different init configs #6184

Silvest89 · 2023-10-10T17:24:10Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Currently the Hetzner provider is very lacking and the autoscaler does not know beforehand which custom taints/labels the provided node pools have.
This PR adds support for a each nodepool to have their own initconfig (currently breaking change, but will see if I have the time to make it backwards compatible). It also supports multi arch node pools. I have been running this a month or so~ build my own image

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Currently this PR introduces a breaking change. However if required I can refactor it so it is backwards compatible

linux-foundation-easycla · 2023-10-10T17:24:16Z

The committers listed above are authorized under a signed CLA.

✅ login: Silvest89 / name: Johnnie Ho (e1408ed)

k8s-ci-robot · 2023-10-10T17:24:19Z

Welcome @Silvest89!

It looks like this is your first PR to kubernetes/autoscaler 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/autoscaler has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

Shubham82 · 2023-10-11T05:19:21Z

Hi @Silvest89
Please sign the CLA before the PR can be reviewed.
See the following document to sign the CLA: Signing Contributor License Agreements(CLA)

Shubham82 · 2023-10-11T05:21:37Z

To check EasyCLA

/easycla

Silvest89 · 2023-10-11T05:25:27Z

Alright, just signed it :)

apricote

Looks very nice 💯

Two things I would like to see before approving this:

Backwards compatibility: We do not want to break users of the older options, so please add HCLOUD_CLOUD_INIT & HCLOUD_IMAGE back in. If one of the old options as well as a the HCLOUD_CLUSTER_CONFIG is defined, I think we should use the newer options.

Docs: It would be great if you can provide an example of how this can be used and how they relate to the older options.

Silvest89 · 2023-10-12T11:16:32Z

Looks very nice 💯

Two things I would like to see before approving this:

Backwards compatibility: We do not want to break users of the older options, so please add HCLOUD_CLOUD_INIT & HCLOUD_IMAGE back in. If one of the old options as well as a the HCLOUD_CLUSTER_CONFIG is defined, I think we should use the newer options.

Docs: It would be great if you can provide an example of how this can be used and how they relate to the older options.

Alright I will be on it. Will update the PR later this week. With updated docs as well :)

Silvest89 · 2023-10-12T19:36:13Z

@apricote
Backwards compatibility and updated docs has been implemented

apricote

Looking good besides one missed error handling and the linting errors from CI :)

https://github.com/kubernetes/autoscaler/actions/runs/6500103013/job/17654801323?pr=6184#step:6:152

apricote · 2023-10-17T06:10:54Z

cluster-autoscaler/cloudprovider/hetzner/hetzner_cloud_provider.go

+		if manager.clusterConfig.IsUsingNewFormat {
+			_, ok := manager.clusterConfig.NodeConfigs[spec.name]
+			if !ok {
+				klog.Fatalf("No node config present for node id `%s` error: %v", spec.name, err)


Suggested change

klog.Fatalf("No node config present for node id `%s` error: %v", spec.name, err)

klog.Fatalf("No node config present for node group id `%s` error: %v", spec.name, err)

apricote · 2023-10-17T06:15:54Z

cluster-autoscaler/cloudprovider/hetzner/hetzner_manager.go

+		if err != nil {
+			return nil, fmt.Errorf("failed to parse cluster config error: %s", err)
+		}
+		json.Unmarshal([]byte(string(clusterConfigEnv)), &clusterConfig)


clusterConfigEnv already is a []byte, so no need to convert to string and back to []byte. https://pkg.go.dev/encoding/base64#Encoding.DecodeString

Suggested change

json.Unmarshal([]byte(string(clusterConfigEnv)), &clusterConfig)

json.Unmarshal(clusterConfigEnv, &clusterConfig)

Also please add error handling for failed json.Unmarshal().

apricote · 2023-10-17T06:17:57Z

cluster-autoscaler/cloudprovider/hetzner/hetzner_node_group.go

 	StartAfterCreate := true
 	opts := hcloud.ServerCreateOpts{
 		Name:             newNodeName(n),
-		UserData:         n.manager.cloudInit,
+		UserData:         string(cloudInit),


cloudInit is already a string

Suggested change

UserData: string(cloudInit),

UserData: cloudInit,

Silvest89 · 2023-10-18T10:27:32Z

@apricote
I've pushed your suggestions! :) Should be alright now

apricote · 2023-10-20T05:44:09Z

/lgtm
/approve

k8s-ci-robot · 2023-10-20T05:44:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: apricote, Silvest89

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cluster-autoscaler/cloudprovider/hetzner/OWNERS~~ [apricote]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

bgervan · 2023-10-28T10:02:56Z

Am I seeing correctly that it is not included in the latest version yet?

Shubham82 · 2023-10-30T09:13:24Z

yes, @bgervan these changes weren't include in the latest version yet i.e. CA v1.28. This feature will be available in CA v1.29.

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 10, 2023

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/provider/hetzner Issues or PRs related to Hetzner provider labels Oct 10, 2023

k8s-ci-robot requested review from apricote and BigDarkClown October 10, 2023 17:24

k8s-ci-robot added the area/cluster-autoscaler label Oct 10, 2023

Silvest89 mentioned this pull request Oct 10, 2023

feat: taints and labels per autoscaler nodepool kube-hetzner/terraform-hcloud-kube-hetzner#1012

Merged

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Oct 11, 2023

apricote suggested changes Oct 12, 2023

View reviewed changes

Silvest89 force-pushed the hetzner-provider-multi-node-pools-support branch from 1c595a3 to 70f6275 Compare October 12, 2023 19:28

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 12, 2023

Silvest89 force-pushed the hetzner-provider-multi-node-pools-support branch from 70f6275 to aa4ea0e Compare October 12, 2023 19:32

Silvest89 requested a review from apricote October 13, 2023 11:52

apricote suggested changes Oct 17, 2023

View reviewed changes

Silvest89 force-pushed the hetzner-provider-multi-node-pools-support branch from aa4ea0e to 412d8e1 Compare October 18, 2023 10:26

feat: each node pool can now have different init configs

e1408ed

Silvest89 force-pushed the hetzner-provider-multi-node-pools-support branch from 412d8e1 to e1408ed Compare October 19, 2023 17:02

Silvest89 requested a review from apricote October 19, 2023 17:46

k8s-ci-robot assigned apricote Oct 20, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 20, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 20, 2023

k8s-ci-robot merged commit f1a5ffe into kubernetes:master Oct 20, 2023
4 checks passed

apricote mentioned this pull request Oct 20, 2023

HETZNER - Custom Labels on scaled nodes #4604

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: each node pool can now have different init configs #6184

feat: each node pool can now have different init configs #6184

Silvest89 commented Oct 10, 2023 •

edited

Loading

linux-foundation-easycla bot commented Oct 10, 2023 •

edited

Loading

k8s-ci-robot commented Oct 10, 2023

Shubham82 commented Oct 11, 2023

Shubham82 commented Oct 11, 2023

Silvest89 commented Oct 11, 2023

apricote left a comment

Silvest89 commented Oct 12, 2023

Silvest89 commented Oct 12, 2023

apricote left a comment

apricote Oct 17, 2023

apricote Oct 17, 2023

apricote Oct 17, 2023

Silvest89 commented Oct 18, 2023

apricote commented Oct 20, 2023

k8s-ci-robot commented Oct 20, 2023

bgervan commented Oct 28, 2023

Shubham82 commented Oct 30, 2023

	klog.Fatalf("No node config present for node id `%s` error: %v", spec.name, err)
	klog.Fatalf("No node config present for node group id `%s` error: %v", spec.name, err)

	json.Unmarshal([]byte(string(clusterConfigEnv)), &clusterConfig)
	json.Unmarshal(clusterConfigEnv, &clusterConfig)

feat: each node pool can now have different init configs #6184

feat: each node pool can now have different init configs #6184

Conversation

Silvest89 commented Oct 10, 2023 • edited Loading

What type of PR is this?

What this PR does / why we need it:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

linux-foundation-easycla bot commented Oct 10, 2023 • edited Loading

k8s-ci-robot commented Oct 10, 2023

Shubham82 commented Oct 11, 2023

Shubham82 commented Oct 11, 2023

Silvest89 commented Oct 11, 2023

apricote left a comment

Choose a reason for hiding this comment

Silvest89 commented Oct 12, 2023

Silvest89 commented Oct 12, 2023

apricote left a comment

Choose a reason for hiding this comment

apricote Oct 17, 2023

Choose a reason for hiding this comment

apricote Oct 17, 2023

Choose a reason for hiding this comment

apricote Oct 17, 2023

Choose a reason for hiding this comment

Silvest89 commented Oct 18, 2023

apricote commented Oct 20, 2023

k8s-ci-robot commented Oct 20, 2023

bgervan commented Oct 28, 2023

Shubham82 commented Oct 30, 2023

Silvest89 commented Oct 10, 2023 •

edited

Loading

linux-foundation-easycla bot commented Oct 10, 2023 •

edited

Loading