Support MixedProtocolLBService and clean up Daemonsets on type change.

[brandond]

Proposed Changes

Support MixedProtocolLBService and clean up Daemonsets on type change.

Also add event support to increase visibility of change events.

Types of Changes

enhancement, compatibility

Verification

See linked issues

Linked Issues

• Container name clash when deploying a mixed protocol load balancer service #4652
• ServiceLB not detecting removed loadbalancers #1870

User-Facing Change

The embedded ServiceLB LoadBalancer controller now supports mixed-protocol Services, and will clean up daemonsets when the Service type changes.

Further Comments

Example kubectl describe with events from installing Traefik and then converting the Service type from LoadBalancer to ClusterIP:

/ # kubectl describe service -n kube-system traefik
Name:              traefik
Namespace:         kube-system
Labels:            app.kubernetes.io/instance=traefik
                   app.kubernetes.io/managed-by=Helm
                   app.kubernetes.io/name=traefik
                   helm.sh/chart=traefik-10.14.100
Annotations:       meta.helm.sh/release-name: traefik
                   meta.helm.sh/release-namespace: kube-system
Selector:          app.kubernetes.io/instance=traefik,app.kubernetes.io/name=traefik
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.43.103.115
IPs:               10.43.103.115
Port:              web  80/TCP
TargetPort:        web/TCP
Endpoints:         10.42.0.5:8000
Port:              websecure  443/TCP
TargetPort:        websecure/TCP
Endpoints:         10.42.0.5:8443
Session Affinity:  None
Events:
  Type    Reason            Age                  From           Message
  ----    ------            ----                 ----           -------
  Normal  UpdatedIngressIP  2m                   svccontroller  LoadBalancer Ingress IP addresses updated: 172.17.0.3
  Normal  UpdatedIngressIP  119s (x2 over 119s)  svccontroller  LoadBalancer Ingress IP addresses updated: 172.17.0.2, 172.17.0.3
  Normal  AppliedDaemonSet  101s (x9 over 2m2s)  svccontroller  Applying LoadBalancer DaemonSet kube-system/svclb-traefik
  Normal  DeletedDaemonSet  34s                  svccontroller  Deleteting DaemonSet kube-system/svclb-traefik

[manuelbuil]

By mixed protocol you mean udp or tcp, right?

[brandond]

@manuelbuil not necessarily; potentially any protocol supported by Kubernetes. I think most if not all only support TCP and UDP though, same as ServiceLB... although Kubernetes itself supports STCP as well.

This is to support a KEP feature that will be enabled by default in 1.24: https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1435-mixed-protocol-lb

[brandond]

Test flaked; merging:

[�[35mPARALLEL�[m] [Fail] [sig-storage] Downward API volume [It] should update labels on modification [NodeConformance] [Conformance] 
[�[35mPARALLEL�[m] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/storage/downwardapi_volume.go:151