Support subordinate user and group IDs provided by SSSD on enterprise FreeIPA set-ups #1074

mhjacks · 2022-07-05T13:52:23Z

Describe the bug

Toolbox currently seems to be hard-coded to look for subuid and subgid in /etc/sub{u,g}id respectively. These ids may now be provided by FreeIPA in a centralized way. Podman and buildah can both take advantage of this new mechanism; maybe toolbox should too

Steps how to reproduce the behaviour

Enable centralized subid support in FreeIPA/IdM (subid: sss in /etc/nsswitch.conf)
Remove /etc/sub{u,g}id entries for the user under test
getsubids <user> should still return a valid subid range
toolbox enter displays an error like this:

$ toolbox enter
Error: /etc/subgid and /etc/subuid don't have entries for user mjackson
See the podman(1), subgid(5), subuid(5) and usermod(8) manuals for more
information.

Expected behaviour
Toolbox should come up and be usable

Actual behaviour
Program reports error and refuses to proceed

Output of toolbox --version (v0.0.90+)
e.g., toolbox version 0.0.90

Toolbox package info (rpm -q toolbox)
toolbox-0.0.99.3-5.fc36.x86_64

Output of podman version

Client:       Podman Engine
Version:      4.1.1
API Version:  4.1.1
Go Version:   go1.18.3
Built:        Wed Jun 15 09:31:58 2022
OS/Arch:      linux/amd64

Podman package info (rpm -q podman)
podman-4.1.1-1.fc36.x86_64

Info about your OS
Fedora Workstation 36

Additional context
We recently troubleshot and fixed essentially the same issue in buildah: containers/buildah#4063

The text was updated successfully, but these errors were encountered:

debarshiray · 2022-08-05T22:47:32Z

Thanks for pointing this out.

We started checking /etc/subgid and /etc/subuid for better error handling on hosts where the current user might have been created long ago with an old version of shadow-utils.

I see that getsubids <user> works on my Fedora 36, even though I don't think I enabled centralized subid support in FreeIPA. I don't see subid: sss in my /etc/nsswitch.conf.

So, I suppose we should first check getsubids <user>. In case the getsubids binary is absent, we can fallback to the directly checking the files. Did I get that right?

mhjacks · 2022-08-06T00:23:48Z

Yes, I think changing the test to use getsubids is an ideal approach - that binary should be present with any version of shadow that supports this scheme. And if the binary is absent, checking the files is as good an approach as any.

debarshiray · 2022-11-30T18:41:23Z

Updated the title of this issue, because @mhjacks has gotten this to work in #1180

Error correctly if no ranges found Add subid-devel dependency for zuul Refactor subid check to move cgo glue elsewhere containers#1074 Signed-off-by: Martin Jackson <[email protected]>

On enterprise FreeIPA set-ups, the subordinate user and group IDs are provided by SSSD's sss plugin for the GNU Name Service Switch (or NSS) functionality of the GNU C Library. They are not listed in /etc/subuid and /etc/subgid. The CGO interaction with libsubid.so are based on 'readSubid' in github.com/containers/storage/pkg/idtools [1]. Unlike 'readSubid', this code considers the absence of any range (ie., nRanges == 0) to be an error as well. Version 4 of the libsubid.so API/ABI [2] was released in Shadow 4.10, which is newer than the versions shipped on RHEL 8 and Debian 9, and even that version had some problems [3]. Therefore, support for older versions, with the relevant workarounds, is necessary. [1] https://github.com/containers/storage/blob/main/pkg/idtools/idtools_supported.go [2] Shadow commit 32f641b207f6ddff shadow-maint/shadow@32f641b207f6ddff [3] Shadow commit 79157cbad87f42cd shadow-maint/shadow@79157cbad87f42cd shadow-maint/shadow#465 containers#1074 Signed-off-by: Martin Jackson <[email protected]>

On enterprise FreeIPA set-ups, the subordinate user and group IDs are provided by SSSD's sss plugin for the GNU Name Service Switch (or NSS) functionality of the GNU C Library. They are not listed in /etc/subuid and /etc/subgid. The CGO interaction with libsubid.so is loosely based on 'readSubid' in github.com/containers/storage/pkg/idtools [1]. Unlike 'readSubid', this code considers the absence of any range (ie., nRanges == 0) to be an error as well. More importantly, this code uses dlopen(3) and friends to dynamically load the symbols from libsubid.so, instead of linking to libsubid.so at build-time and having the dependency noted in the /usr/bin/toolbox binary. This is done because libsubid.so itself depends on several other shared libraries, and indirect dependencies can't be influenced by the RUNPATH [2] embedded in the /usr/bin/toolbox binary [3]. Hence, when the binary is used inside Toolbx containers (eg., as the entry point), those indirect dependencies won't be picked from the host's runtime against which the binary was built. This can render the binary useless due to ABI compatibility issues. Using dlopen(3) avoids this problem, especially because libsubid.so is only used when running on the host. Version 4 of the libsubid.so API/ABI [4] was released in Shadow 4.10, which is newer than the versions shipped on RHEL 8 and Debian 10 [5], and even that newer version had some problems [6]. Therefore, support for older versions, with the relevant workarounds, is necessary. This code doesn't set the public variables Prog and shadow_logfd that older Shadow versions used to expect for logging, because from Shadow 4.9 onwards there's a separate function [4,7] to specify these. This can be changed if there are libsubid.so versions in the wild that really do need those public variables to be set. Finally, ISO C99 is required because of the use of <stdbool.h> in the libsubid.so API. Some changes by Debarshi Ray. [1] https://github.com/containers/storage/blob/main/pkg/idtools/idtools_supported.go [2] https://man7.org/linux/man-pages/man8/ld.so.8.html [3] Commit 6063eb2 containers#821 [4] Shadow commit 32f641b207f6ddff shadow-maint/shadow@32f641b207f6ddff shadow-maint/shadow#443 [5] https://packages.debian.org/source/buster/shadow [6] Shadow commit 79157cbad87f42cd shadow-maint/shadow@79157cbad87f42cd shadow-maint/shadow#465 [7] Shadow commit 2b22a6909dba60d shadow-maint/shadow@2b22a6909dba60d shadow-maint/shadow#325 containers#1074 Signed-off-by: Martin Jackson <[email protected]>

On enterprise FreeIPA set-ups, the subordinate user and group IDs are provided by SSSD's sss plugin for the GNU Name Service Switch (or NSS) functionality of the GNU C Library. They are not listed in /etc/subuid and /etc/subgid. The CGO interaction with libsubid.so is loosely based on 'readSubid' in github.com/containers/storage/pkg/idtools [1]. Unlike 'readSubid', this code considers the absence of any range (ie., nRanges == 0) to be an error as well. More importantly, this code uses dlopen(3) and friends to dynamically load the symbols from libsubid.so, instead of linking to libsubid.so at build-time and having the dependency noted in the /usr/bin/toolbox binary. This is done because libsubid.so itself depends on several other shared libraries, and indirect dependencies can't be influenced by the RUNPATH [2] embedded in the /usr/bin/toolbox binary [3]. Hence, when the binary is used inside Toolbx containers (eg., as the entry point), those indirect dependencies won't be picked from the host's runtime against which the binary was built. This can render the binary useless due to ABI compatibility issues. Using dlopen(3) avoids this problem, especially because libsubid.so is only used when running on the host. Care was taken to not load and link libsubid.so twice to separately validate the subordinate ranges for the user and the group. Sadly, there doesn't seem to be a way to close the file descriptor that's used by libsubid.so for logging. Hence, this one file descriptor (currently number 3, unless the parent process passes down others) pointing to /dev/null is leaked for the life cycle of the toolbox process. Version 4 of the libsubid.so API/ABI [4] was released in Shadow 4.10, which is newer than the versions shipped on RHEL 8 and Debian 10 [5], and even that newer version had some problems [6]. Therefore, support for older versions, with the relevant workarounds, is necessary. This code doesn't set the public variables Prog and shadow_logfd that older Shadow versions used to expect for logging, because from Shadow 4.9 onwards there's a separate function [4,7] to specify these. This can be changed if there are libsubid.so versions in the wild that really do need those public variables to be set. Finally, ISO C99 is required because of the use of <stdbool.h> in the libsubid.so API. Some changes by Debarshi Ray. [1] https://github.com/containers/storage/blob/main/pkg/idtools/idtools_supported.go [2] https://man7.org/linux/man-pages/man8/ld.so.8.html [3] Commit 6063eb2 containers#821 [4] Shadow commit 32f641b207f6ddff shadow-maint/shadow@32f641b207f6ddff shadow-maint/shadow#443 [5] https://packages.debian.org/source/buster/shadow [6] Shadow commit 79157cbad87f42cd shadow-maint/shadow@79157cbad87f42cd shadow-maint/shadow#465 [7] Shadow commit 2b22a6909dba60d shadow-maint/shadow@2b22a6909dba60d shadow-maint/shadow#325 containers#1074 Signed-off-by: Martin Jackson <[email protected]>

On enterprise FreeIPA set-ups, the subordinate user and group IDs are provided by SSSD's sss plugin for the GNU Name Service Switch (or NSS) functionality of the GNU C Library. They are not listed in /etc/subuid and /etc/subgid. Therefore, its necessary to use libsubid.so to check the subordinate ID ranges. The CGO interaction with libsubid.so is loosely based on 'readSubid' in github.com/containers/storage/pkg/idtools [1]. Unlike 'readSubid', this code considers the absence of any range (ie., nRanges == 0) to be an error as well. More importantly, this code uses dlopen(3) and friends to dynamically load the symbols from libsubid.so, instead of linking to libsubid.so at build-time and having the dependency noted in the /usr/bin/toolbox binary. This is done because libsubid.so itself depends on several other shared libraries, and indirect dependencies can't be influenced by the RUNPATH [2] embedded in the /usr/bin/toolbox binary [3]. Hence, when the binary is used inside Toolbx containers (eg., as the entry point), those indirect dependencies won't be picked from the host's runtime against which the binary was built. This can render the binary useless due to ABI compatibility issues. Using dlopen(3) avoids this problem, especially because libsubid.so is only used when running on the host. Care was taken to not load and link libsubid.so twice to separately validate the subordinate ID ranges for the user and the group. Sadly, there doesn't seem to be a way to close the file descriptor that's used by libsubid.so for logging. Hence, this one file descriptor (currently number 3, unless the parent process passes down others) pointing to /dev/null is leaked for the life cycle of the toolbox process. Version 4 of the libsubid.so API/ABI [4] was released in Shadow 4.10, which is newer than the versions shipped on RHEL 8 and Debian 10 [5], and even that newer version had some problems [6]. Therefore, support for older versions, with the relevant workarounds, is necessary. This code doesn't set the public variables Prog and shadow_logfd that older Shadow versions used to expect for logging, because from Shadow 4.9 onwards there's a separate function [4,7] to specify these. This can be changed if there are libsubid.so versions in the wild that really do need those public variables to be set. Finally, ISO C99 is required because of the use of <stdbool.h> in the libsubid.so API. Some changes by Debarshi Ray. [1] https://github.com/containers/storage/blob/main/pkg/idtools/idtools_supported.go [2] https://man7.org/linux/man-pages/man8/ld.so.8.html [3] Commit 6063eb2 containers#821 [4] Shadow commit 32f641b207f6ddff shadow-maint/shadow@32f641b207f6ddff shadow-maint/shadow#443 [5] https://packages.debian.org/source/buster/shadow [6] Shadow commit 79157cbad87f42cd shadow-maint/shadow@79157cbad87f42cd shadow-maint/shadow#465 [7] Shadow commit 2b22a6909dba60d shadow-maint/shadow@2b22a6909dba60d shadow-maint/shadow#325 containers#1074 Signed-off-by: Martin Jackson <[email protected]>

The Toolbx source code was being repeatedly re-built with 'go run ...' to generate each of the shell completions. A following commit will use CGO to link to libsubid.so, which will sufficiently complicate the build that a simple 'go run ...' won't be enough. Hence, it's better to use the same mechanisms in src/go-build-wrapper that are used to generate the Toolbx binary. However, the main Toolbx binary only works if /run/host exists [1], which is only created after 'meson install' and may not exist during 'meson compile'. This is solved by using a throwaway binary that lacks the ABI protections necessary to run as a container's entry, but is enough to generate the shell completions. Fallout from bafbbe8 [1] Commit 6063eb2 containers#821 containers#1074

On enterprise FreeIPA set-ups, the subordinate user and group IDs are provided by SSSD's sss plugin for the GNU Name Service Switch (or NSS) functionality of the GNU C Library. They are not listed in /etc/subuid and /etc/subgid. Therefore, its necessary to use libsubid.so to check the subordinate ID ranges. The CGO interaction with libsubid.so is loosely based on 'readSubid' in github.com/containers/storage/pkg/idtools [1]. Unlike 'readSubid', this code considers the absence of any range (ie., nRanges == 0) to be an error as well. More importantly, this code uses dlopen(3) and friends to dynamically load the symbols from libsubid.so, instead of linking to libsubid.so at build-time and having the dependency noted in the /usr/bin/toolbox binary. This is done because libsubid.so itself depends on several other shared libraries, and indirect dependencies can't be influenced by the RUNPATH [2] embedded in the /usr/bin/toolbox binary [3]. Hence, when the binary is used inside Toolbx containers (eg., as the entry point), those indirect dependencies won't be picked from the host's runtime against which the binary was built. This can render the binary useless due to ABI compatibility issues. Using dlopen(3) avoids this problem, especially because libsubid.so is only used when running on the host. Care was taken to not load and link libsubid.so twice to separately validate the subordinate ID ranges for the user and the group. Sadly, there doesn't seem to be a way to close the file descriptor that's used by libsubid.so for logging. Hence, this one file descriptor (currently number 3, unless the parent process passes down others) pointing to /dev/null is leaked for the life cycle of the toolbox process. Version 4 of the libsubid.so API/ABI [4] was released in Shadow 4.10, which is newer than the versions shipped on RHEL 8 and Debian 10 [5], and even that newer version had some problems [6]. Therefore, support for older versions, with the relevant workarounds, is necessary. This code doesn't set the public variables Prog and shadow_logfd that older Shadow versions used to expect for logging, because from Shadow 4.9 onwards there's a separate function [4,7] to specify these. This can be changed if there are libsubid.so versions in the wild that really do need those public variables to be set. Finally, ISO C99 is required because of the use of <stdbool.h> in the libsubid.so API. Some changes by Debarshi Ray. [1] https://github.com/containers/storage/blob/main/pkg/idtools/idtools_supported.go [2] https://man7.org/linux/man-pages/man8/ld.so.8.html [3] Commit 6063eb2 containers#821 [4] Shadow commit 32f641b207f6ddff shadow-maint/shadow@32f641b207f6ddff shadow-maint/shadow#443 [5] https://packages.debian.org/source/buster/shadow [6] Shadow commit 79157cbad87f42cd shadow-maint/shadow@79157cbad87f42cd shadow-maint/shadow#465 [7] Shadow commit 2b22a6909dba60d shadow-maint/shadow@2b22a6909dba60d shadow-maint/shadow#325 containers#1074 Signed-off-by: Martin Jackson <[email protected]>

On enterprise FreeIPA set-ups, the subordinate user and group IDs are provided by SSSD's sss plugin for the GNU Name Service Switch (or NSS) functionality of the GNU C Library. They are not listed in /etc/subuid and /etc/subgid. Therefore, its necessary to use libsubid.so to check the subordinate ID ranges. The CGO interaction with libsubid.so is loosely based on 'readSubid' in github.com/containers/storage/pkg/idtools [1]. However, unlike 'readSubid', this code considers the absence of any range (ie., nRanges == 0) to be an error as well. More importantly, this code uses dlopen(3) and friends to dynamically load the symbols from libsubid.so, instead of linking to libsubid.so at build-time and having the dependency noted in the /usr/bin/toolbox binary. This is done because libsubid.so itself depends on several other shared libraries, and indirect dependencies can't be influenced by the RUNPATH [2] embedded in the /usr/bin/toolbox binary [3]. Hence, when the binary is used inside Toolbx containers (eg., as the entry point), those indirect dependencies won't be picked from the host's runtime against which the binary was built. This can render the binary useless due to ABI compatibility issues. Using dlopen(3) avoids this problem, especially because libsubid.so is only used when running on the host. Care was taken to not load and link libsubid.so twice to separately validate the subordinate ID ranges for the user and the group. Note that libsubid_init() must be passed a FILE pointer for logging. Otherwise, it will create it's own for logging, and there's no way to close it during dlclose(3). Version 4 of the libsubid.so API/ABI [4] was released in Shadow 4.10, which is newer than the versions shipped on RHEL 8 and Debian 10 [5], and even that newer version had some problems [6]. Therefore, support for older versions, with the relevant workarounds, is necessary. Fortunately, the oldest that needs to be support is Shadow 4.9 because that's when libsubid.so was introduced [7]. Note that SUBID_ABI_VERSION was only introduced with version 4 of the libsubid.so API/ABI released in Shadow 4.10 [8]. The first release of libsubid.so in Shadow 4.9 already had an ABI version of 3.0.0 [9], since it was bumped a few times during development, so that's what's assumed when SUBID_ABI_VERSION is absent. This code doesn't set the public variables Prog and shadow_logfd that older Shadow versions used to expect for logging, because from Shadow 4.9 onwards there's a separate function [4,10] to specify these. This can be changed if there are libsubid.so versions in the wild that really do need those public variables to be set. Finally, ISO C99 is required because of the use of <stdbool.h> in the libsubid.so API. Some changes by Debarshi Ray. [1] https://github.com/containers/storage/blob/main/pkg/idtools/idtools_supported.go [2] https://man7.org/linux/man-pages/man8/ld.so.8.html [3] Commit 6063eb2 containers#821 [4] Shadow commit 32f641b207f6ddff shadow-maint/shadow@32f641b207f6ddff shadow-maint/shadow#443 [5] https://packages.debian.org/source/buster/shadow [6] Shadow commit 79157cbad87f42cd shadow-maint/shadow@79157cbad87f42cd shadow-maint/shadow#465 [7] Shadow commit 0a7888b1fad613a0 shadow-maint/shadow@0a7888b1fad613a0 shadow-maint/shadow#154 [8] Shadow commit 0c9f64140852e8d5 shadow-maint/shadow@0c9f64140852e8d5 shadow-maint/shadow#449 [9] Shadow commit 3d670ba7ed58f910 shadow-maint/shadow@3d670ba7ed58f910 shadow-maint/shadow#339 [10] Shadow commit 2b22a6909dba60d shadow-maint/shadow@2b22a6909dba60d shadow-maint/shadow#325 containers#1074 Signed-off-by: Martin Jackson <[email protected]>

debarshiray · 2023-01-28T21:23:39Z

This was fixed in #1180 (and #1219).

Now that Toolbx offers built-in support for Ubuntu containers [1], adding an Ubuntu host to the upstream CI will help ensure that Toolbx continues to work well on Ubuntu. Ubuntu 22.04 is the latest long term support (or LTS) release [2] from Ubuntu, and is the latest Ubuntu version that GitHub provides runners for [3]. Ubuntu 22.04 only has Bats 1.2.1 [4], while Toolbx requires 1.7.0 [5]; and Shadow 4.8 [6], while Toolbx requires 4.9 because it needs libsubid.so [7,8]. Hence, newer versions of these dependencies need to be built to run the tests. The build flags for Shadow were taken from the Debian package [9]. A separate sub-directory inside $GITHUB_WORKSPACE [10] is used for Toolbx itself to prevent codespell from getting triggered by spelling mistakes in these dependencies themselves [11]. Unfortunately, the SHELL environment variable goes mysteriously missing from the runtime environment of the GitHub Actions workflow [12]. This breaks the 'create' and 'enter' commands, and therefore tests involving them can't be run until this is resolved. Meanwhile, running the CI on Ubuntu with a subset of the tests, is still better than not running the CI on Ubuntu at all. [1] Commit a84a358 containers#483 containers#1284 [2] https://wiki.ubuntu.com/Releases [3] https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners [4] https://packages.ubuntu.com/jammy/bats [5] Commit e22a82f containers#1273 [6] https://packages.ubuntu.com/source/jammy/shadow https://packages.ubuntu.com/source/jammy-updates/shadow [7] Shadow commit 0a7888b1fad613a0 shadow-maint/shadow@0a7888b1fad613a0 shadow-maint/shadow#154 [8] Commit ca8007c containers#1074 [9] https://salsa.debian.org/debian/shadow/ [10] https://docs.github.com/en/actions/learn-github-actions/variables [11] bats-core/bats-core#743 [12] https://github.com/orgs/community/discussions/59413 containers#1319

See: containers/toolbox@ca8007c192b3bb77 containers/toolbox#1074 containers#35 Signed-off-by: Penn Bauman <[email protected]>

mhjacks added the 1. Bug Something isn't working label Jul 5, 2022

infra-monkey mentioned this issue Oct 4, 2022

[BUG] FreeIPA Enterprise sign-in fails to map subUID and subGID fedora-silverblue/issue-tracker#263

Closed

mhjacks mentioned this issue Nov 30, 2022

Enhance subid check by using shadow library function #1180

Closed

debarshiray changed the title ~~Should toolbox switch to new idMapping mechanism for mapping subids?~~ Support subordinate user and group IDs provided by SSSD on enterprise FreeIPA set-ups Nov 30, 2022

debarshiray mentioned this issue Nov 30, 2022

RFE: Make toolbox-init-container a fully static binary or disable CGO to avoid compatibility issues with container's glibc #832

Closed

mhjacks mentioned this issue Nov 30, 2022

Enhance subid check by using library function #1184

Closed

debarshiray mentioned this issue Jan 28, 2023

Support subordinate user and group ID ranges on enterprise set-ups #1219

Merged

debarshiray closed this as completed Jan 28, 2023

Steve973 mentioned this issue May 4, 2023

setting /etc/subuid and /etc/subgid doesn't work well on enterprise systems that use FreeIPA mgoltzsche/podman-static#61

Closed

debarshiray mentioned this issue Jun 27, 2023

.github/workflows: Run the CI on Ubuntu 22.04 #1319

Merged

debarshiray pushed a commit to pennbauman/containertoolbx.org that referenced this issue Sep 27, 2023

Add shadow to the list of build dependencies

c594aff

See: containers/toolbox@ca8007c192b3bb77 containers/toolbox#1074 containers#35 Signed-off-by: Penn Bauman <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support subordinate user and group IDs provided by SSSD on enterprise FreeIPA set-ups #1074

Support subordinate user and group IDs provided by SSSD on enterprise FreeIPA set-ups #1074

mhjacks commented Jul 5, 2022

debarshiray commented Aug 5, 2022

mhjacks commented Aug 6, 2022

debarshiray commented Nov 30, 2022

debarshiray commented Jan 28, 2023

Support subordinate user and group IDs provided by SSSD on enterprise FreeIPA set-ups #1074

Support subordinate user and group IDs provided by SSSD on enterprise FreeIPA set-ups #1074

Comments

mhjacks commented Jul 5, 2022

debarshiray commented Aug 5, 2022

mhjacks commented Aug 6, 2022

debarshiray commented Nov 30, 2022

debarshiray commented Jan 28, 2023