This Terraform plan is for creating a kube-proxy -less multi-node RKE2 cluster using Cilium CNI. The Cilium chart enables BBR Pod Congestion Control for greatly increased network performance as well. Also installed is Rancher's Prometheus Operator Chart (cluster_monitoring.tf) along with an HAProxy Ingress Grafana Dashboard to gain visibility into your Ingress traffic (this plan disables the built-in RKE2 NGiNX Ingress in favor of HAProxy because...reasons ¯\_(ツ)_/¯ ). kube-vip makes a token appearance here to advertise the HAProxy Ingress Controller via ARP.
This is a nice boiler-plate Terraform plan for a high-performing cluster that includes a very capable monitoring stack.
-
Functional Rancher Management Server with vSphere Cloud Credential
-
vCenter >= 7.x and credentials with appropriate permissions (see https://github.com/rancher/barn/blob/main/Walkthroughs/vSphere/Permissions/README.md)
-
Virtual Machine Hardware Compatibility at Version >= 15
-
Linux Kernel >= 5.18 (required for enabling BBR Pod Congestion Control)
-
Create the following in the files/ directory:
NAME PURPOSE .rancher-api-url URL for Rancher Management Server .rancher-bearer-token API bearer token generated via Rancher UI .ssh-public-key SSH public key for additional OS user
-
Cilium's Hubble UI is disabled as it can be a drag on performance. However, if you enjoy looking at groupings of rectangles connected with lines and do want to enable Hubble, reference the RKE2 Cilium Helm chart HERE.
-
If you don't want to run the Rancher Prometheus Operator, it can be uninstalled at any time simply by removing
cluster_monitoring.tffrom your working directory and re-applying the plan. It is here as a demonstration/value-add, not a requirement of any kind. -
kube-vip is operating via ARP mode, so services published via LoadBalancer will have traffic directed to a single node.
-
Unlike RKE2's "baked-in" NGiNX Ingress Controller, HAProxy's ingress is not FIPS 140-2 compliant.
terraform apply| SOFTWARE | VERSION | DOCS |
|---|---|---|
| kube-vip | 0.6.2 | https://kube-vip.io/docs |
| Rancher Prometheus Operator | 102.0.2+up40.1.2 | https://docs.ranchermanager.rancher.io/pages-for-subheaders/monitoring-and-alerting |
| Rancher Server | 2.7.6 | https://rancher.com/docs/rancher/v2.6/en/overview |
| Rancher Terraform Provider | 3.1.1 | https://registry.terraform.io/providers/rancher/rancher2/latest/docs |
| RKE2 | 1.26.8+rke2r1 | https://docs.rke2.io |
| Terraform | 1.4.6 | https://www.terraform.io/docs |
| vSphere | 8.0.1.00300 | https://docs.vmware.com/en/VMware-vSphere/index.html |