Glibc support for Firecracker

[aliguori]

This allows the standard rust toolchain to work out of the box. This makes Firecracker much more accessible for a typical user while still retaining the ability to build for musl.

[alxiord]

I think we need to wait a bit before changing the default target to glibc because:

• main reason: when activated, seccomp filters will break. glibc is a lot more invasive than musl in terms of syscalls and Firecracker's filters are built as tight as possible, tailored around musl. I don't consider relaxing the filters to accommodate glibc an option - we can support both glibc and musl and maybe build a set of filters for glibc and make it clear that this doesn't go well with our security posture when we release them, but we can't let users build with glibc and tell them to enable seccomp simultaneously yet.
• side reason: musl came into question in the first place because the previously dynamically glibc-linked Firecracker crashed in certain situations because of libc version conflicts, and static linking against glibc is not a viable option. Firecracker was supposed to "just work" out of the box on any system. Now that the focus shifts from "just works" to "just builds", glibc support seems unavoidable, but I believe the binary we ship in the release should be the same as the binary you get with the default cargo build.

[aliguori]

Will refactor this to avoid breaking jailer

[dhrgit]

I think defaulting to glibc needs a more in-depth discussion.

cargo build would work out of the box with the musl target (without the need to have musl libc installed on the host system), if it weren't for the backtrace crate, which has an external C dependency - libbacktrace. That's even if libbacktrace doesn't depend on any particular libc flavor (it's compiled with no-std).

It's actually the linking of libbacktrace that fails, and only if it's compiled with GCC. It links just fine if compiled with clang. That's because GCC defaults to -D_FORTIFY_SOURCE, relying on glibc specific support for stack protection. musl uses a fully-inline approach to achieve the same result, so no specific support is needed in the object code.

Now, if built without -D_FORTIFY_SOURCE, even against the glibc headers, libbacktrace links just fine, and firecracker builds perfectly and passes all the integration tests.

Does building (this particular lib only) without -D_FORTIFY_SOURCE introduce any kind of risk?

[aliguori]

Do we really need to depend on libbacktrace?

…

On Mon, Nov 26, 2018, 12:18 PM Dan Horobeanu ***@***.*** wrote: I think defaulting to glibc needs a more in-depth discussion. cargo build would work out of the box with the musl target (without the need to have musl libc installed on the host system), if it weren't for the backtrace crate, which has an external C dependency - libbacktrace <https://github.com/rust-lang-nursery/libbacktrace>. That's even if libbacktrace doesn't depend on any particular libc flavor (it's compiled with no-std). It's actually the linking of libbacktrace that fails, and only if it's compiled with GCC. It links just fine if compiled with clang. That's because GCC defaults to -D_FORTIFY_SOURCE, relying on glibc specific support for stack protection. musl uses a fully-inline approach <https://wiki.musl-libc.org/open-issues.html#Substitute-for-%3Ccode%3E_FORTIFY_SOURCE%3C/code%3E> to achieve the same result, so no specific support is needed in the object code. Now, if built without -D_FORTIFY_SOURCE, even against the glibc headers, libbacktrace links just fine, and firecracker builds perfectly and passes all the integration tests. Does building (this particular lib only) without -D_FORTIFY_SOURCE introduce any kind of risk? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#647 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABGYoQ4_gN-NH9J4ZsOEosTjmb8b1GK5ks5uzEyHgaJpZM4YyGFy> .

[dhrgit]

Do we really need to depend on libbacktrace?

We currently use a stack trace lib to dump the trace into the log, in case of an unexpected panic.

We couldn't find a rust-native alternative to libbacktrace. And, if it's C, chances are GCC will generate some of those stack protection calls.

Barring a rust stack trace solution (or a similarly helpful troubleshooting mechanism), these are the options I can think of, in the order of my preference:

• use devtool for building
  drawback: requires Docker
• or compile libbacktrace with -N_FORTIFY_SOURCE
  drawback: need to investigate impact
• or default to glibc
  drawbacks:
  • invalidates our static linking, just-works model
  • requires a more relaxed seccomp filter
• or don't log stack trace on panic
  drawback: makes troubleshooting considerably more difficult

[messense]

We couldn't find a rust-native alternative to libbacktrace.

FYI, there is an ongoing effort to replace libbacktrace with a native Rust implementation in rustc

[andreeaflorescu]

+1 for using a rust-native alternative to libbacktrace

[andreeaflorescu]

Closing this due to no activity, please re-open if you still want to get it merge.