calico-node Pod gets in to CrashLoopBackOff state on IPv6 Only node

[mdrahman-suse]

Environmental Info:
RKE2 Version:

rke2 -v
rke2 version v1.32.0-rc2+rke2r1 (c9d57c6a33180a2693fa5813c96f465864a73b38)
go version go1.23.3 X:boringcrypto

Also observed the same with v1.31.4+rke2r1

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"

Cluster Configuration:

1 server

Describe the bug:

On an IPv6 only node, when rke2 is installed, the pod calico-node goes in to CrashLoopBackOff state

Steps To Reproduce:

• Pre installation
• Update /etc/hosts

::1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback i-02358e76052a733fb

• Update /etc/resolv.conf

nameserver 2a00:1098:2c::1
options edns0 trust-ad
search us-west-1.compute.internal

• /etc/rancher/rke2/config.yaml

write-kubeconfig-mode: 644
cni: calico
cluster-cidr: 2001:cafe:42:0::/56
service-cidr: 2001:cafe:43:0::/112
node-ip: <ipv6>
disable-network-policy: true
flannel-ipv6-masq: true

• Installed RKE2: curl -sfL https://get.rke2.io | sudo INSTALL_RKE2_VERSION=v1.32.0-rc2+rke2r1 sh -

Expected behavior:

Pods should be either in Running or Completed state

Actual behavior:

Pod calico-node is in state

Additional context / logs:

• Pods

$ kgp
NAMESPACE         NAME                                                    READY   STATUS             RESTARTS      AGE
calico-system     calico-kube-controllers-656c5d6d48-qqmrh                1/1     Running            0             4m27s
calico-system     calico-node-tvsf5                                       0/1     CrashLoopBackOff   5 (63s ago)   4m28s
calico-system     calico-typha-74f9645b88-8btb5                           1/1     Running            0             4m36s
kube-system       cloud-controller-manager-i-02358e76052a733fb            1/1     Running            0             6m15s
kube-system       etcd-i-02358e76052a733fb                                1/1     Running            0             6m15s
kube-system       helm-install-rke2-calico-crd-6ngvr                      0/1     Completed          0             6m21s
kube-system       helm-install-rke2-calico-n5czc                          0/1     Completed          2             6m21s
kube-system       helm-install-rke2-coredns-l9bqm                         0/1     Completed          0             6m21s
kube-system       helm-install-rke2-ingress-nginx-hjxcf                   0/1     Completed          0             6m21s
kube-system       helm-install-rke2-metrics-server-fp9gr                  0/1     Completed          0             6m21s
kube-system       helm-install-rke2-snapshot-controller-crd-jfmcz         0/1     Completed          0             6m21s
kube-system       helm-install-rke2-snapshot-controller-csbs5             0/1     Completed          0             6m20s
kube-system       helm-install-rke2-snapshot-validation-webhook-q5j4f     0/1     Completed          0             6m20s
kube-system       kube-apiserver-i-02358e76052a733fb                      1/1     Running            0             6m15s
kube-system       kube-controller-manager-i-02358e76052a733fb             1/1     Running            0             6m15s
kube-system       kube-proxy-i-02358e76052a733fb                          1/1     Running            0             6m15s
kube-system       kube-scheduler-i-02358e76052a733fb                      1/1     Running            0             6m15s
kube-system       rke2-coredns-rke2-coredns-7895c6f5d8-fss94              1/1     Running            0             6m4s
kube-system       rke2-coredns-rke2-coredns-autoscaler-5868d76f68-plhn9   1/1     Running            0             6m4s
kube-system       rke2-ingress-nginx-controller-nlxxk                     1/1     Running            0             4m1s
kube-system       rke2-metrics-server-55f5b7fcd8-bwp9p                    1/1     Running            0             4m51s
kube-system       rke2-snapshot-controller-755fc86fc-mg54n                1/1     Running            0             4m47s
kube-system       rke2-snapshot-validation-webhook-75bc6f574f-xt8sk       1/1     Running            0             4m56s
tigera-operator   tigera-operator-7c46f8dfc7-cm449                        1/1     Running            0             5m48s

• Pod logs

$ k logs -n calico-system calico-node-tvsf5
Defaulted container "calico-node" out of: calico-node, flexvol-driver (init), install-cni (init)
2024-12-25 00:20:21.032 [INFO][10] startup/startup.go 439: Early log level set to info
2024-12-25 00:20:21.033 [INFO][10] startup/utils.go 126: Using NODENAME environment for node name i-02358e76052a733fb
2024-12-25 00:20:21.033 [INFO][10] startup/utils.go 138: Determined node name: i-02358e76052a733fb
2024-12-25 00:20:21.033 [INFO][10] startup/startup.go 95: Starting node i-02358e76052a733fb with version v3.29.1
2024-12-25 00:20:21.034 [INFO][10] startup/startup.go 444: Checking datastore connection
2024-12-25 00:20:21.048 [INFO][10] startup/startup.go 468: Datastore connection verified
2024-12-25 00:20:21.053 [INFO][10] startup/startup.go 105: Datastore is ready
2024-12-25 00:20:21.083 [INFO][10] startup/customresource.go 102: Error getting resource Key=GlobalFelixConfig(name=CalicoVersion) Name="calicoversion" Resource="GlobalFelixConfigs" error=the server could not find the requested resource (get GlobalFelixConfigs.crd.projectcalico.org calicoversion)
2024-12-25 00:20:21.094 [WARNING][10] startup/winutils.go 150: Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
2024-12-25 00:20:21.109 [INFO][10] startup/startup.go 497: Initialize BGP data
2024-12-25 00:20:21.110 [WARNING][10] startup/autodetection_methods.go 99: Unable to auto-detect an IPv4 address: no valid IPv4 addresses found on the host interfaces
2024-12-25 00:20:21.110 [WARNING][10] startup/startup.go 519: Couldn't autodetect an IPv4 address. If auto-detecting, choose a different autodetection method. Otherwise provide an explicit address.
2024-12-25 00:20:21.110 [INFO][10] startup/startup.go 403: Clearing out-of-date IPv4 address from this node IP=""
2024-12-25 00:20:21.111 [INFO][10] startup/startup.go 407: Clearing out-of-date IPv6 address from this node IP=""
2024-12-25 00:20:21.129 [WARNING][10] startup/utils.go 48: Terminating
Calico node failed to start

[brandond]

According to https://docs.tigera.io/calico/latest/networking/ipam/ipv6 ipv6 only operation requires customizing the ippool configuration via chart values.

To configure an IPv6-only cluster using the operator, edit your default Installation at install time to include a single IPv6 pool, and no IPv4 pools.

I am not sure how IP pools are wired up to the cluster CIDRs in our charts, but I suspect this will require use of a HelmChartConfig resources.