Operator fails reconciling database spec with LoadBalancerSourceRanges specified

[gwvandesteeg]

Report

When attempting to reconcile a chart defined with

    haproxy:
      enabled: true
      exposePrimary:
        enabled: true
        type: LoadBalancer
        loadBalancerSourceRanges:
          - 10.0.0.0/8

The reconciliation fails.

More about the problem

Pod logs

2024-12-05T04:10:19.467Z        ERROR   Reconciler error        {"controller": "pxc-controller", "namespace": "default", "name": "mysql-pxc-db", "reconcileID": "1215ff06-2c03-4a92-b8f7-97eca57198c9", "error": "mysql-pxc-db-haproxy-replicas upgrade error: Service \"mysql-pxc-db-haproxy-replicas\" is invalid: spec.LoadBalancerSourceRanges: Forbidden: may only be used when `type` is 'LoadBalancer'", "errorVerbose": "Service \"mysql-pxc-db-haproxy-replicas\" is invalid: spec.LoadBalancerSourceRanges: Forbidden: may only be used when `type` is 'LoadBalancer'\nmysql-pxc-db-haproxy-replicas upgrade error\ngithub.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).reconcileHAProxy\n\t/go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/controller.go:515\ngithub.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc.(*ReconcilePerconaXtraDBCluster).Reconcile\n\t/go/src/github.com/percona/percona-xtradb-cluster-operator/pkg/controller/pxc/controller.go:366\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1695"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222

Steps to reproduce

1. Create chart with a valid loadBalancerSourceRanges specified
2. Update the chart by changing something simple like an annotation
3. Poof, failure.

Versions

1. Kubernetes:

$ kubectl version
Client Version: v1.31.2
Kustomize Version: v5.4.2
Server Version: v1.31.2-eks-7f9249a

2. Operator (Fails with both charts for v1.15.0 and v1.15.1)

$ kubectl logs -n pxc-operator mysql-operator-bc8dcc85d-zwtr7 -f
2024-12-05T01:18:51.956Z        INFO    setup   Runs on {"platform": "kubernetes", "version": "v1.31.2-eks-7f9249a"}
2024-12-05T01:18:51.956Z        INFO    setup   Manager starting up     {"gitCommit": "9807f08b1c46ffdf141fb9935e36bb09cd7edefe", "gitBranch": "release-1-15-1", "buildTime": "2024-10-14T17:02:29Z", "goVersion": "go1.22.8", "os": "linux", "arch": "amd64"}

3. Database (pxc chart version: v1.15.0)

image: percona/percona-xtradb-cluster:8.0.36-28.1

Anything else?

This is not the only thing broken upon reconciliation, with a reconciliation failure backup tasks/jobs don't get created/updated.

[gwvandesteeg]

Appears to be related to this code:

percona-xtradb-cluster-operator/pkg/pxc/service.go

Lines 429 to 444 in a7a036d

	if cr.CompareVersionWith("1.14.0") >= 0 {
	if cr.Spec.HAProxy.ExposeReplicas.ServiceExpose.LoadBalancerSourceRanges != nil {
	loadBalancerSourceRanges = cr.Spec.HAProxy.ExposeReplicas.ServiceExpose.LoadBalancerSourceRanges
	} else {
	loadBalancerSourceRanges = cr.Spec.HAProxy.ExposePrimary.LoadBalancerSourceRanges
	}
	loadBalancerIP = cr.Spec.HAProxy.ExposeReplicas.ServiceExpose.LoadBalancerIP
	} else {
	if cr.Spec.HAProxy.ReplicasLoadBalancerSourceRanges != nil {
	loadBalancerSourceRanges = cr.Spec.HAProxy.ReplicasLoadBalancerSourceRanges
	} else {
	loadBalancerSourceRanges = cr.Spec.HAProxy.LoadBalancerSourceRanges
	}
	loadBalancerIP = cr.Spec.HAProxy.ReplicasLoadBalancerIP
	}
	}

[hors]

@gwvandesteeg thank you! I have create jira task for this bug https://perconadev.atlassian.net/browse/K8SPXC-1538