-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: enclave-cc improvement ideas #235
Comments
Hm. I don't have the full picture, but I think you're right that the second proposal is a bit like the nydus approach that we are adding elsewhere in the project. The nydus snapshotter is untrusted and we rely on some additional artifacts for verification/decryption. The first proposal is very interesting. If the image management runs at a node level, it would be shared between different pods or maybe even different clients. This seems a little weird, but I guess the idea is that each pod would be able to verify the evidence of the image management enclave and the API would be designed in a way to limit privilege. I wonder if this approach could make sense for other runtime classes as well? I'm not sure how the overhead would compare to the untrusted snapshotter ideas. |
I'm not familiar with enclave-cc architecture, but the proposal seems reasonable.
The proposal 1 is actually providing a node-level filesystem server with encryption for enclave-cc. |
The current idea is to use sealing (#149). Originally we talked about some LA-attested channel between the two but setting this up between a snapshotter and a pod is tricky. Moreover, SGX provides a flexible mechanism to protect data between enclaves signed by the same author. With the snapshotter (run in an enclave), runtime metadata (entrypoint, envs) and the per image encrypted fs key can be sealed. The per image "init" can be trusted to use that information only and start the app using the runtime metadata provided. The open is how to trust this runtime metadata but at least it could use the image's |
I gave an enclave-cc Q3 23' update and talked about improvement ideas in the Sept 14 community call (the last ~18minutes or so).
I'll summarize the improvement ideas here so that we can follow-up with detailed discussion.
enclave-cc existing challenges follow the "Kata-CC" challenges:
Additionally, enclave-cc implementation is sub-optimal with non-encrypted containers since it uses Occlum encrypted sefs even if it wasn't strictly necessary.
So far we have successfully demonstrated that unmodified containers can be made libOS aware by mounting a "libOS layer" during runtime. I'm calling this "overlayos". :-) With the existing setup we are also using CoCo KBS/KBC to attest enclave-agent to get access to layer encryption secrets.
In the today's architecture,
enclave-agent
is responsible for preparing the application image "bundle" andshim-rune
is responsible for creating the overlayos mount of the application image + "Occlum libOS layer" (installed on each SGX enabled host by CoCo operator).From Kata-CC we have learned that image layers can be processed by a containerd "remote snapshotter". It's also possible to make the snapshotter configuration per runtime handler.
Proposal 1. - encrypted containers without a forked containerd
a. move per pod
enclave-agent
to a per node snapshotter run in an Occlum TEEb. drop existing
shim-rune
(the Occlum TEE snapshotter providesmounts
info toshim-runc
to process)c. #18 can be mitigated by
socat
to translate/tmp/tee-snapshotter.sock
to a format Occlum understandsd. explore if Occlum
hostfs
mounts can be leveraged to store unpacked layers for improved layer sharingProposal 2. - unencrypted containers with user-defined policies + Gramine
To address #73, the problem of intermediate file encryption when non-encrypted images are processed, and the layer sharing problems, a slightly different approach is proposed. It still follows the "overlayos" approach but by using Gramine libOS and its built-in policy/manifest mechanism.
a. have a non-TEE snapshotter that handles image layers the same way as the containerd built-in overlayfs snapshotter but adds Gramine LibOS layer on top (installed on each host by the CoCo operator)
b. user provided manifest + enclave signature is either bind mounted from a
configMap
or pulled by the snapshotter from an OCI v1.1 enabled registry (with OCI reference types it's possible to build a link between the manifest+enclave signature blob and the image they are for).The user provided manifest approach follows the same needs as already approved for Kata-CC as the "container metadata validation".
Summary:
Extras:
The two proposals keep enclave-cc compatible with the CoCo goals where can unmodified containers run in TEEs. To further tighten the link between "proposal 2" and use of CoCo components, I've built a demonstrator showing how Proposal 2. uses CoCo KBS protocol and KBS to enable k8s sealed secrets for enclave-cc:
The text was updated successfully, but these errors were encountered: