feature: Support sandbox configuration option for FUSE

[jbleonesio]

As discussed with @juergbi BuildStream is lacking a way to tune the build sandbox configuration.

For example, it is possible to expose some devices to the shell sandbox (eg. /dev/fuse) but not on the build sandbox unfortunately.

For my personal use case I wanted to extract the content of a MacOS .dmg file using a remote element (using darling-dmg bundled as a dependency) but was quickly stopped by the lack of /dev/fuse exposition in the build sandbox.

I'm willing to give a try to start implementing something if someone can give me some pointers/context.

[juergbi]

I'm willing to give a try to start implementing something if someone can give me some pointers/context.

For local execution, this should be possible to support without any changes in BuildBox. BuildStream would need to append --bind-mount=/dev/fuse:/dev/fuse to the command constructed here, if the new config option is specified.

The sandbox config structure is handled here.

Ideally, we'd also support this for remote execution. However, this would require defining a platform property and adding support for that in buildbox-run-bubblewrap, which is a part of https://gitlab.com/BuildGrid/buildbox/buildbox/. The platform property approach should work for both remote and local execution, so this would not require the --bind-mount option mentioned above.

[jbleonesio]

Simply passing --bind-mount=/dev/fuse:/dev/fuse in _sandboxbuildboxrun.py doesn't seem to make the cut since the error message I get passed from fuse: device not found, try 'modprobe fuse' first to fusermount: mount failed: Operation not permitted.

I can however confirm that /dev/fuse is properly exposed in the build sandbox :

+ sh -c -e 'ls -lh /dev
'
total 0
lrwxrwxrwx 1     0     0      11 Nov 27 17:44 core -> /proc/kcore
lrwxrwxrwx 1     0     0      13 Nov 27 17:44 fd -> /proc/self/fd
crw-rw-rw- 1 65534 65534  1,   7 Nov 26 12:44 full
crw-rw-rw- 1 65534 65534 10, 229 Nov 26 12:44 fuse
crw-rw-rw- 1 65534 65534  1,   3 Nov 26 12:44 null
lrwxrwxrwx 1     0     0       8 Nov 27 17:44 ptmx -> pts/ptmx
drwxr-xr-x 2     0     0       0 Nov 27 17:44 pts
crw-rw-rw- 1 65534 65534  1,   8 Nov 26 12:44 random
drwxr-xr-x 2     0     0      40 Nov 27 17:44 shm
lrwxrwxrwx 1     0     0      15 Nov 27 17:44 stderr -> /proc/self/fd/2
lrwxrwxrwx 1     0     0      15 Nov 27 17:44 stdin -> /proc/self/fd/0
lrwxrwxrwx 1     0     0      15 Nov 27 17:44 stdout -> /proc/self/fd/1
crw-rw-rw- 1 65534 65534  5,   0 Nov 27 15:12 tty
crw-rw-rw- 1 65534 65534  1,   9 Nov 26 12:44 urandom
crw-rw-rw- 1 65534 65534  1,   5 Nov 26 12:44 zero

I guess the build sandbox also needs some additional capabilities to be able to use the mount syscall.

[abderrahim]

Yeah, you need CAP_SYS_ADMIN to be able to mount things.

I kinda feel that fuse isn't the right thing to use in a sandbox

[juergbi]

I don't think there is anything conceptually wrong with using FUSE in a sandbox. However, I would certainly avoid the problem space if there is an alternative, and requiring CAP_SYS_ADMIN makes it a more problematic.

Since Linux 4.18, the namespace-restricted CAP_SYS_ADMIN should suffice but it's still a significant increase in kernel API surface and I think, if we were to support it, adding CAP_SYS_ADMIN should require an explicit, separate platform property, not implicitly be enabled via a FUSE option. It also means that it won't be possible to support FUSE without changes in BuildBox. Changing buildbox-run-bubblewrap to not pass --cap-drop ALL to bwrap might do the trick.

@jbleonesio Have you considered using 7z or the library provided by darling-dmg to extract the .dmg instead of FUSE?

[jbleonesio]

I understand that managing capabilities in the sandbox is not an inconsequential matter.

@juergbi Luckily, I just managed to extract the content of the .dmg using dmg2img to decompress the dmg file prior to extract it with 7z because 7z alone wasn't able to extract the compressed dmg (that was the trick).

It's valuable to have a way to extract dmg files in BuildStream build sandbox because this format is a widespread way to distribute precompiled software for MacOS.

In some other situations were no alternatives to mounting filesystems could be found, having the opportunity to use fuse would still be valuable IMHO.

[nanonyme]

Would it be complete nonsense to handle this case through source plugin? So host does the FUSE mount to unpack data into source cache and the content gets imported into sandbox for further processing.