[BUG] Docker Containers are not encrypted

[flikites]

Docker containers that are currently deployed on flux are not one-way encrypted. The Docker containers deployed onto Flux nodes can currently be manipulated by the Flux node operator.

[bunnyanalyst]

I did explore solutions for this issue. I don't think there is any bulletproof way to solve this problem as long as node operators still have access to the computer. Typically, research papers focus on the fact that the codes are potentially malicious and the host are honest actors, not the other way round (i.e., hosts are malicious and codes are honest). Docker and other Sandbox environments are designed based on the fact that hosts are honest actors.

Because of that design decision, I found no practical way to stop the node operator from SSH into the docker containers. There are some workarounds (e.g., deleting /bin/sh and /bin/bash program in the container, obfuscating codes), but it does not seem like a way to solve the problem technically.

The best way to solve this issue is by detection and penalty. If the Flux benchmark can be updated to detect malicious behavior and kick them out of the queue when malicious behaviors are detected, that should be a good enough solution.

[flikites]

I like your proposed approach about detecting and penalizing. It seems like "deleting /bin/sh and /bin/bash program in the container, obfuscating codes" would take away the ability for the app owner to use those commands as well? How good of a detection program can be built?How easy would such a program be bypassed by the node operator?  ---- On Wed, 23 Nov 2022 01:08:37 -0500 ***@***.*** wrote ---- I did explore solutions for this issue. I don't think there is any bulletproof way to solve this problem as long as node operators still have access to the computer. Typically, research papers focus on the fact that the codes are potentially malicious and the host are honest actors, not the other way round (i.e., hosts are malicious and codes are honest). Docker and other Sandbox environments are designed based on the fact that hosts are honest actors. Because of that design decision, I found no practical way to stop the node operator from SSH into the docker containers. There are some workarounds (e.g., deleting /bin/sh and /bin/bash program in the container, obfuscating codes), but it does not seem like a way to solve the problem technically. The best way to solve this issue is by detection and penalty. If the Flux benchmark can be updated to detect malicious behavior and kick them out of the queue when malicious behaviors are detected, that should be a good enough solution. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[bunnyanalyst]

Those methods are always just workaround, and we will play hide and seek with node operators if that are the methods we use to solve the issue. In addition, removing /bin/bash and /bin/sh always means that software deployers will lose the feature to execute shell script remotely (since most useful programs are removed).

Code obfuscation only means that it is difficult for a node operator to read the codes, but they can always change the codes completely if they wanted to.

[w2vy]

There is clearly no quick fix for this.

If we could introduce a Operational and Maintenance Mode for the Node and then not allow root access when in Operational mode (ie only time apps can run)

We are renting the whole machine and require nothing else to be allowed to run.
We'd also need to change the root password and give node op sudo access only in Maintenance Mode

[MorningLightMountain713]

There is clearly no quick fix for this.

If we could introduce a Operational and Maintenance Mode for the Node and then not allow root access when in Operational mode (ie only time apps can run)

We are renting the whole machine and require nothing else to be allowed to run. We'd also need to change the root password and give node op sudo access only in Maintenance Mode

This. Removing sudo rights from node ops is the only valid security measure to ensure privacy for app owners. Clealy permit rules would be needed for host security updates and maintenance, reboot etc. This leads to a full custom OS for Flux as a solution. Or, as an interum, give up sudo rights as part of install from multitoolbox or something.

Is it worth pursuing other avenues in the meatime?

The way I allways look at it - whatever obfusication I'm doing, there are allways ways of getting the data. Would I be happy putting ETH validation keys on there? No, okay.

[cmdntd]

Could this solved with encrypt private image at v4.0.0?

Applications that target nodes are called enteprise and get access to private images, secrets which is achieved by encrypting app specifications.

https://github.com/RunOnFlux/flux/releases/tag/v4.0.0

[flikites]

Not really, that data encryption is decrypted once it hits the node. Therefore the container would run in an unencrypted fashion after it has spun up. We need to be able to spin up an encrypted container without the node operator being able to get to the password that encrypted the container. :) ---- On Sat,30 Sep 2023 09:26:46 -0400 ***@***.*** wrote ---- Could this solved with encrypt private image at v4.0.0? Applications that target nodes are called enteprise and get access to private images, secrets which is achieved by encrypting app specifications. https://github.com/RunOnFlux/flux/releases/tag/v4.0.0 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>