-
Notifications
You must be signed in to change notification settings - Fork 20
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into enable-metrics
- Loading branch information
Showing
8 changed files
with
193 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
# Cilium BGP Load-Balancing | ||
|
||
This flavor creates special labeled worker nodes for ingress which leverage Cilium's | ||
[BGP Control Plane](https://docs.cilium.io/en/stable/network/bgp-control-plane/) | ||
and [LB IPAM](https://docs.cilium.io/en/stable/network/lb-ipam/) support. | ||
|
||
With this flavor, Services exposed via `type: LoadBalancer` automatically get | ||
assigned an `ExternalIP` provisioned as a shared IP through the | ||
[linode-CCM](https://github.com/linode/linode-cloud-controller-manager?tab=readme-ov-file#shared-ip-load-balancing), | ||
which is deployed with the necessary settings to perform shared IP load-balancing. | ||
|
||
```admonish warning | ||
There are a couple important caveats to load balancing support based on current | ||
Linode networking and API limitations: | ||
1. **Ingress traffic will not be split between BGP peer nodes** | ||
[Equal-Cost Multi-Path (ECMP)](https://en.wikipedia.org/wiki/Equal-cost_multi-path_routing) | ||
is not supported on the BGP routers so ingress traffic will not be split between each | ||
BGP Node in the cluster. One Node will be actively receiving traffic and the other(s) | ||
will act as standby(s). | ||
2. **Customer support is required to use this feature at this time** | ||
Since this uses additional IPv4 addresses on the nodes participating in Cilium's | ||
BGPPeeringPolicy, you need to [contact our Support team](https://www.linode.com/support/) | ||
to be permitted to add extra IPs. | ||
``` | ||
|
||
```admonish note | ||
Dual-stack support is enabled for clusters using this flavor since IPv6 is used for router | ||
and neighbor solicitation. | ||
Without enabling dual-stack support, the IPv6 traffic is blocked if the Cilium host firewall | ||
is enabled (which it is by default in CAPL), even if there are no configured `CiliumClusterWideNetworkPolicies` | ||
or the policy is set to audit (default) instead of enforce (see [https://github.com/cilium/cilium/issues/27484](https://github.com/cilium/cilium/issues/27484)). More information about firewalling can be found on the [Firewalling](../firewalling.md) page. | ||
``` | ||
|
||
## Specification | ||
|
||
| Control Plane | CNI | Default OS | Installs ClusterClass | IPv4 | IPv6 | | ||
|---------------|--------|--------------|-----------------------|------|------| | ||
| Kubeadm | Cilium | Ubuntu 22.04 | No | Yes | Yes | | ||
|
||
|
||
## Prerequisites | ||
|
||
1. [Quickstart](../getting-started.md) completed | ||
|
||
## Usage | ||
|
||
1. (Optional) Set up environment variable | ||
```sh | ||
# Optional | ||
export BGP_PEER_MACHINE_COUNT=2 | ||
``` | ||
|
||
2. Generate cluster yaml | ||
|
||
```sh | ||
clusterctl generate cluster test-cluster \ | ||
--kubernetes-version v1.29.1 \ | ||
--infrastructure linode-linode \ | ||
--flavor kubeadm-cilium-bgp-lb > test-cluster.yaml | ||
``` | ||
|
||
3. Apply cluster yaml | ||
|
||
```sh | ||
kubectl apply -f test-cluster.yaml | ||
``` | ||
|
||
After the cluster exists, you can create a Service exposed with `type: LoadBalancer` and | ||
it will automatically get assigned an ExternalIP. It's recommended to set up an ingress controller | ||
(e.g. [https://docs.cilium.io/en/stable/network/servicemesh/ingress/](https://docs.cilium.io/en/stable/network/servicemesh/ingress/)) | ||
to avoid needing to expose multiple `LoadBalancer` Services within the cluster. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 changes: 5 additions & 0 deletions
5
templates/flavors/kubeadm/cilium-bgp-lb/kubeadmConfigTemplate.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1 | ||
kind: KubeadmConfigTemplate | ||
metadata: | ||
name: ${CLUSTER_NAME}-md-bgp |
96 changes: 96 additions & 0 deletions
96
templates/flavors/kubeadm/cilium-bgp-lb/kustomization.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
apiVersion: kustomize.config.k8s.io/v1beta1 | ||
kind: Kustomization | ||
resources: | ||
- ../dual-stack | ||
- machineDeployment.yaml | ||
- linodeMachineTemplate.yaml | ||
- kubeadmConfigTemplate.yaml | ||
|
||
patches: | ||
- target: | ||
kind: HelmChartProxy | ||
name: .*-linode-cloud-controller-manager | ||
patch: |- | ||
- op: replace | ||
path: /spec/valuesTemplate | ||
value: | | ||
sharedIPLoadBalancing: | ||
loadBalancerType: cilium-bgp | ||
bgpNodeSelector: cilium-bgp-peering=true | ||
secretRef: | ||
name: "linode-token-region" | ||
image: | ||
pullPolicy: IfNotPresent | ||
env: | ||
- name: LINODE_URL | ||
value: https://api.linode.com/v4beta | ||
transformers: | ||
- | | ||
apiVersion: builtin | ||
kind: ReplacementTransformer | ||
metadata: | ||
name: clone-bgp | ||
replacements: | ||
- source: | ||
kind: KubeadmConfigTemplate | ||
name: ${CLUSTER_NAME}-md-0 | ||
fieldPath: .spec | ||
targets: | ||
- select: | ||
kind: KubeadmConfigTemplate | ||
name: ${CLUSTER_NAME}-md-bgp | ||
fieldPaths: | ||
- .spec | ||
options: | ||
create: true | ||
- source: | ||
kind: LinodeMachineTemplate | ||
name: ${CLUSTER_NAME}-md-0 | ||
fieldPath: .spec | ||
targets: | ||
- select: | ||
kind: LinodeMachineTemplate | ||
name: ${CLUSTER_NAME}-md-bgp | ||
fieldPaths: | ||
- .spec | ||
options: | ||
create: true | ||
- source: | ||
kind: MachineDeployment | ||
name: ${CLUSTER_NAME}-md-0 | ||
fieldPath: .spec.template | ||
targets: | ||
- select: | ||
kind: MachineDeployment | ||
name: ${CLUSTER_NAME}-md-bgp | ||
fieldPaths: | ||
- .spec.template | ||
options: | ||
create: true | ||
- |- | ||
apiVersion: builtin | ||
kind: PatchTransformer | ||
metadata: | ||
name: patch-kubeadmconfig | ||
patch: |- | ||
- op: add | ||
path: /spec/template/spec/joinConfiguration/nodeRegistration/kubeletExtraArgs/node-labels | ||
value: cilium-bgp-peering="true" | ||
target: | ||
kind: KubeadmConfigTemplate | ||
name: .*-md-bgp | ||
- |- | ||
apiVersion: builtin | ||
kind: PatchTransformer | ||
metadata: | ||
name: patch-machinedeployment | ||
patch: |- | ||
- op: replace | ||
path: /spec/template/spec/bootstrap/configRef/name | ||
value: ${CLUSTER_NAME}-md-bgp | ||
- op: replace | ||
path: /spec/template/spec/infrastructureRef/name | ||
value: ${CLUSTER_NAME}-md-bgp | ||
target: | ||
kind: MachineDeployment | ||
name: .*-md-bgp |
4 changes: 4 additions & 0 deletions
4
templates/flavors/kubeadm/cilium-bgp-lb/linodeMachineTemplate.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1 | ||
kind: LinodeMachineTemplate | ||
metadata: | ||
name: ${CLUSTER_NAME}-md-bgp |
8 changes: 8 additions & 0 deletions
8
templates/flavors/kubeadm/cilium-bgp-lb/machineDeployment.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
apiVersion: cluster.x-k8s.io/v1beta1 | ||
kind: MachineDeployment | ||
metadata: | ||
name: ${CLUSTER_NAME}-md-bgp | ||
spec: | ||
clusterName: ${CLUSTER_NAME} | ||
replicas: ${BGP_PEER_MACHINE_COUNT:=2} |