do we need to run buildah containers always with BUILDAH_ISOLATION = chroot

[himmatss]

Hi,

I have a buildah container image (quay.io/buildah/stable:latest) running with default setting as a "BUILDAH_ISOLATION = chroot" in Kubernetes. However, I am wondering is this really required to run the buildah as a container ?

Can someone pleas explain this ,
https://github.com/containers/buildah/blob/main/docs/buildah-build.1.md
_"--isolation type

Controls what type of isolation is used for running processes as part of RUN instructions. Recognized types include oci (OCI-compatible runtime, the default), rootless (OCI-compatible runtime invoked using a modified configuration, with --no-new-keyring added to its create invocation, reusing the host's network and UTS namespaces, and creating private IPC, PID, mount, and user namespaces; the default for unprivileged users), and chroot (an internal wrapper that leans more toward chroot(1) than container technology, reusing the host's control group, network, IPC, and PID namespaces, and creating private mount and UTS namespaces, and creating user namespaces only when they're required for ID mapping).

Note: You can also override the default isolation type by setting the BUILDAH_ISOLATION environment variable. export BUILDAH_ISOLATION=oci"_

[nalind]

In many cases, a container that's run using the image will not be given enough privileges for buildah run or the handling of RUN instructions in Dockerfiles in buildah build to be able to launch a container using an actual runtime like crun or runc. The chroot-based method is severely limited in functionality compared to crun or runc, but in return it exercises fewer privileges than they might, so it works (or "works") in a number of cases where they might not. If your environment provides enough privileges to not have to use chroot, feel free to override it.

[himmatss]

Thanks @nalind for your reply.
The documentation says the default value is "oci" for the BUILDAH_ISOLATION but in the dockerfile of the image quay.io/buildah/stable:latest ; it appears to be having the BUILDAH_ISOLATION=chroot
https://github.com/containers/image_build/blob/main/podman/Containerfile
https://github.com/containers/image_build/blob/main/buildah/Containerfile

[nalind]

Yes, the container image has the environment variable set in it to override the compiled-in default.

[chmeliik]

I have a similar need to run buildah in Kubernetes with better isolation.

If your environment provides enough privileges to not have to use chroot, feel free to override it.

What privileges are those? How can I check if the environment provides them?

[nalind]

For handling RUN instructions, it's a combination of

• Being able to create a user namespace with multiple IDs mapped into it, or being started as UID 0 and having CAP_SYS_ADMIN, so that it doesn't need to do those things to set up a namespace where those things are true. If you're writing the pod spec, hostUsers: false may provide some of this.
• Being able to create bind and overlay mounts for volumes (this generally requires CAP_SYS_ADMIN) that it provides.
• Being able to chroot into the rootfs to make changes inside of it (CAP_SYS_CHROOT).
• Being able to configure networking for a namespace that it creates if "host" networking isn't specified. There's no reason to not use "host" networking when we're in a container, because from buildah's point of view, the container's network is the host network, but that's configurable, and the hard-coded defaults don't assume being run inside of a container.
• Being able to successfully execute a command using runc, or crun, or a comparable runtime that can be invoked similarly. That last part introduces some requirements of its own that we don't have control over.

Some of these operations can also be denied by the seccomp filter, or by the SELinux policy (or other mandatory access control rules), and it's entirely possible that I'm still forgetting some things. For me, it tends to be a trial-and-error process.