Enable Cilium node-to-node strict encryption #2462
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for constellation-docs canceled.
|
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
2 times, most recently
from
8fb4d36 to
e80aa81
Compare
3u13r
changed the title
This was referenced Oct 19, 2023
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
5 times, most recently
from
5462998 to
8618331
Compare
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
from
8618331 to
2aae003
Compare
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
from
2aae003 to
77fe878
Compare
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
2 times, most recently
from
7b27ee4 to
2565c8c
Compare
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
4 times, most recently
from
2db3cdb to
0d4d3c3
Compare
daniel-weisse
approved these changes
Nov 15, 2023
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
from
2e620a1 to
9d1de1e
Compare
|
Reverting back to draft to figure out migration off of konnectivity. See broken upgrade test. |
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
3 times, most recently
from
707e16c to
1da5bc4
Compare
Bumping Cilium to also enable node-to-node encryption and node-to-node strict mode. Since the second is not upstream we use our fork.
When enabling node-to-node encryption, Cilium does not encrypt control-plane to control-plane traffic by default since they say that they cannot gurantee that the generated private key for a node is persisted across reboots. In Constellation we use stateful VMs which when rebooted still have the cilium_wg0 interface containing the private key. Therefore, we can enable this type of encryption.
For the strict modes we need to dynamically use the CIDR used in the Terraform files. Therefore, we write them to our statefile and use them when installing Cilium.
The token given out by control-planes contains the node IP as an endpoint. Since during this stage the joining node is not connected to the WireGuard network, we cannot communicate node-to-node. Therefore, we need to hop over the load balancer again to have a src IP outside of the strict range.
Use the local variable instead of inlining the node CIDR value.
The Cilium strict mode has a special mode which loosens the security a slight bit. For compatability this mode is enabled by default. But we don't need it for strict node-to-node encryption. Therefore, we disable it.
Tests concluded that restating the Cilium agent after the first boot is not needed anymore to regain connectivity for pods.
This is the first step in our migration off of konnectivity. Before node-to-node encryption we used konnectivity to route some KubeAPI to kubelet traffic over the pod network which then would be encrypted. Since we enabled node-to-node encryption this has no security upsides anymore. Note that we still deploy the konnectivity agents via helm and still have the load balancer for konnectivity. In the following releases we will remove both.
3u13r
force-pushed
the
feat/cilium/strict-node-to-node
branch
from
576e22f to
d5a9f9f
Compare
Coverage report
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Proposed change(s)
FixCilium(). There is no need to restart the agent anymore.Additional info
Minicon: https://github.com/edgelesssys/constellation/actions/runs/6864239262 (This fails due to the main nightly image still setting up konnectivity, when using the image from this branch the test works:
ref/feat-cilium-strict-node-to-node/stream/nightly/v2.13.0-pre.0.20231114141207-0d4d3c3d8386)e2e, azure, lb: https://github.com/edgelesssys/constellation/actions/runs/6864247842
e2e, gcp, verify: https://github.com/edgelesssys/constellation/actions/runs/6864253985
e2e, aws, autoscaling: https://github.com/edgelesssys/constellation/actions/runs/6864259968
e2e, upgrade, gcp 3:2, from v2.12.0: https://github.com/edgelesssys/constellation/actions/runs/6877667384
Again:
e2e, azure, lb: https://github.com/edgelesssys/constellation/actions/runs/6878967924
e2e, gcp, verify: https://github.com/edgelesssys/constellation/actions/runs/6878976138
e2e, aws, autoscaling: https://github.com/edgelesssys/constellation/actions/runs/6878980995
e2e, upgrade, gcp 3:2, from v2.12.0: https://github.com/edgelesssys/constellation/actions/runs/6878954935
Checklist