Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't run rootless Podman with users managed by systemd-homed on a correct setup. #20040

Closed
IPlayZed opened this issue Sep 19, 2023 · 8 comments
Closed

Can't run rootless Podman with users managed by systemd-homed on a correct setup. #20040

IPlayZed opened this issue Sep 19, 2023 · 8 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@IPlayZed
Copy link

IPlayZed commented Sep 19, 2023 •
edited
Loading

Issue Description

Command podman run -v -dt -p 8080:80/tcp docker.io/library/httpd fails with:

Trying to pull docker.io/library/httpd:latest...
Getting image source signatures
Copying blob 8c226ac2053e done  
Copying blob 2832a695827e done  
Copying blob 360eba32fa65 done  
Copying blob 45a0ea29816d done  
Copying blob b57c1299d233 done  
Error: copying system image from manifest list: writing blob: adding layer with blob "sha256:360eba32fa65016e0d558c6af176db31a202e9a6071666f9b629cb8ba6ccedf0": processing tar file(potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/gshadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/gshadow: invalid argument): exit status 1

Steps to reproduce the issue

Steps to reproduce the issue

  1. Use a systemd-homed managed user.
  2. Setup rootless Podman.
  3. Try to run a Podman container, like podman run -v -dt -p 8080:80/tcp docker.io/library/httpd.

Describe the results you received

Describe the results you received

Describe the results you expected

Container runs normally as it does launching it as a superuser.

podman info output

Podman info:

❯ podman info
host:
  arch: amd64
  buildahVersion: 1.31.2
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: /usr/bin/conmon is owned by conmon 1:2.1.8-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: 00e08f4a9ca5420de733bf542b930ad58e1a7e7d'
  cpuUtilization:
    idlePercent: 93.49
    systemPercent: 5.87
    userPercent: 0.64
  cpus: 32
  databaseBackend: boltdb
  distribution:
    distribution: arch
    version: unknown
  eventLogger: journald
  freeLocks: 2048
  hostname: minefpc
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 60311
      size: 1
    uidmap:
    - container_id: 0
      host_id: 60311
      size: 1
  kernel: 6.4.15-hardened1-1-hardened
  linkmode: dynamic
  logDriver: journald
  memFree: 117821149184
  memTotal: 134976802816
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: Unknown
    package: /usr/lib/podman/netavark is owned by netavark 1.7.0-1
    path: /usr/lib/podman/netavark
    version: netavark 1.7.0
  ociRuntime:
    name: crun
    package: /usr/bin/crun is owned by crun 1.9-1
    path: /usr/bin/crun
    version: |-
      crun version 1.9
      commit: a538ac4ea1ff319bcfe2bf81cb5c6f687e2dc9d3
      rundir: /run/user/60311/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: /usr/bin/pasta is owned by passt 2023_09_08.05627dc-1
    version: |
      pasta unknown version
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    path: /run/user/60311/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: /usr/bin/slirp4netns is owned by slirp4netns 1.2.2-1
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.4
  swapFree: 164024020992
  swapTotal: 164982943744
  uptime: 0h 37m 8.00s
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /home/personal/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/personal/.local/share/containers/storage
  graphRootAllocated: 279153827840
  graphRootUsed: 108797673472
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /run/user/60311/containers
  transientStore: false
  volumePath: /home/personal/.local/share/containers/storage/volumes
version:
  APIVersion: 4.6.2
  Built: 1693343961
  BuiltTime: Tue Aug 29 23:19:21 2023
  GitCommit: 5db42e86862ef42c59304c38aa583732fd80f178-dirty
  GoVersion: go1.21.0
  Os: linux
  OsArch: linux/amd64
  Version: 4.6.2

Podman version:

❯ podman version
Client:       Podman Engine
Version:      4.6.2
API Version:  4.6.2
Go Version:   go1.21.0
Git Commit:   5db42e86862ef42c59304c38aa583732fd80f178-dirty
Built:        Tue Aug 29 23:19:21 2023
OS/Arch:      linux/amd64

Kernel:

❯ uname -a
Linux minefpc 6.4.15-hardened1-1-hardened #1 SMP PREEMPT_DYNAMIC Tue, 12 Sep 2023 17:08:22 +0000 x86_64 GNU/Linux

OS: Arch Linux (x64)

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

The output of userdbctl:

NAME                           DISPOSITION        UID   GID REALNAME                         HOME                     SHELL
   root                           intrinsic            0     0 -                                /root                    /bin/bash
┌─ ↓ begin system users ↓         system               1     - First system user                -                        -
   bin                            system               1     1 -                                /                        /usr/bin/nologin
   daemon                         system               2     2 -                                /                        /usr/bin/nologin
   mail                           system               8    12 -                                /var/spool/mail          /usr/bin/nologin
   ftp                            system              14    11 -                                /srv/ftp                 /usr/bin/nologin
   rpc                            system              32    32 Rpcbind Daemon                   /var/lib/rpcbind         /usr/bin/nologin
   http                           system              33    33 -                                /srv/http                /usr/bin/nologin
   clamav                         system              64    64 Clam AntiVirus                   /                        /usr/bin/nologin
   uuidd                          system              68    68 -                                /                        /usr/bin/nologin
   dbus                           system              81    81 System Message Bus               /                        /usr/bin/nologin
   polkitd                        system             102   102 PolicyKit daemon                 /                        /usr/bin/nologin
   gdm                            system             120   120 Gnome Display Manager            /var/lib/gdm             /usr/bin/nologin
   rtkit                          system             133   133 RealtimeKit                      /proc                    /usr/bin/nologin
   usbmux                         system             140   140 usbmux user                      /                        /usr/bin/nologin
   nvidia-persistenced            system             143   143 NVIDIA Persistence Daemon        /                        /usr/bin/nologin
   cups                           system             209   209 cups helper user                 /                        /usr/bin/nologin
   fwupd                          system             951   951 Firmware update daemon           /var/lib/fwupd           /usr/bin/nologin
   systemd-journal-upload         system             953   953 systemd Journal Upload           /                        /usr/bin/nologin
   saned                          system             955   955 SANE daemon user                 /                        /usr/bin/nologin
   mysql                          system             956   956 MariaDB                          /var/lib/mysql           /usr/bin/nologin
   sddm                           system             958   958 Simple Desktop Display Manager   /var/lib/sddm            /usr/bin/nologin
   nm-openconnect                 system             960   960 NetworkManager OpenConnect       /                        /usr/bin/nologin
   openvpn                        system             961   961 OpenVPN                          /                        /usr/bin/nologin
   nm-openvpn                     system             962   962 NetworkManager OpenVPN           /                        /usr/bin/nologin
   dnsmasq                        system             963   963 dnsmasq daemon                   /                        /usr/bin/nologin
   qemu                           system             964   964 QEMU user                        /                        /usr/bin/nologin
   libvirt-qemu                   system             965   965 Libvirt QEMU user                /                        /usr/bin/nologin
   gluster                        system             966   966 GlusterFS daemons                /var/run/gluster         /usr/bin/nologin
   git                            system             968   968 git daemon user                  /                        /usr/bin/git-shell
   geoclue                        system             969   969 Geoinformation service           /var/lib/geoclue         /usr/bin/nologin
   flatpak                        system             970   970 Flatpak system helper            /                        /usr/bin/nologin
   colord                         system             971   971 Color management daemon          /var/lib/colord          /usr/bin/nologin
   brltty                         system             972   972 Braille Device Daemon            /var/lib/brltty          /usr/bin/nologin
   avahi                          system             973   973 Avahi mDNS/DNS-SD daemon         /                        /usr/bin/nologin
   tss                            system             975   975 tss user for tpm2                /                        /usr/bin/nologin
   systemd-timesync               system             976   976 systemd Time Synchronization     /                        /usr/bin/nologin
   systemd-resolve                system             977   977 systemd Resolver                 /                        /usr/bin/nologin
   systemd-journal-remote         system             978   978 systemd Journal Remote           /                        /usr/bin/nologin
   systemd-oom                    system             979   979 systemd Userspace OOM Killer     /                        /usr/bin/nologin
   systemd-network                system             980   980 systemd Network Management       /                        /usr/bin/nologin
   systemd-coredump               system             981   981 systemd Core Dumper              /                        /usr/bin/nologin
└─ ↑ end system users ↑           system             999     - Last system user                 -                        -
   developer-no-homed             regular           1000  1000 developer (not Homed managed)    /home/developer-no-homed /bin/bash
┌─ ↓ begin systemd-homed users ↓  regular          60001     - First systemd-homed user         -                        -
   gaming                         regular          60197 60197 Börcsök Balázs Róbert (gaming)   /home/gaming             /bin/bash
   admin                          regular          60282 60282 Börcsök Balázs Róbert (admin)    /home/admin              /bin/bash
   personal                       regular          60311 60311 Börcsök Balázs Róbert (personal) /home/personal           /bin/bash
└─ ↑ end systemd-homed users ↑    regular          60513     - Last systemd-homed user          -                        -
┌─ ↓ begin mapped users ↓         regular          60514     - First mapped user                -                        -
└─ ↑ end mapped users ↑           regular          60577     - Last mapped user                 -                        -
┌─ ↓ begin dynamic system users ↓ dynamic          61184     - First dynamic system user        -                        -
└─ ↑ end dynamic system users ↑   dynamic          65519     - Last dynamic system user         -                        -
   nobody                         intrinsic        65534 65534 Kernel Overflow User             /                        /usr/bin/nologin
┌─ ↓ begin container users ↓      container       524288     - First container user             -                        -
└─ ↑ end container users ↑        container   1878982656     - Last container user              -                        -
46 users listed.

It works fine for users not managed by systemd-homed.
@IPlayZed IPlayZed added the kind/bug Categorizes issue or PR as related to a bug. label Sep 19, 2023
@mheon
Copy link
Member

mheon commented Sep 19, 2023

I believe systemd-homed uses NFS, no? If so, I think this is expected as NFS does not support subuid/subgid ranges now. @giuseppe Does that sound correct?

@IPlayZed
Copy link
Author

I believe systemd-homed uses NFS, no? If so, I think this is expected as NFS does not support subuid/subgid ranges now. @giuseppe Does that sound correct?

What do you mean by NFS? The filesystem the user's home directory is mapped to is Btrfs, if that is the concern.

@mheon
Copy link
Member

mheon commented Sep 19, 2023

Ah, looks like that's not necessarily a requirement.

Googling about suggests that you could be seeing something related to systemd/systemd#21952 (or, alternatively, systemd-homed has not configured subuid/subgid ranges for the users in question at all?)

@rhatdan
Copy link
Member

rhatdan commented Sep 19, 2023

The problem is around the /etc/subuid and /etc/subgid fields not being populated.

@giuseppe
Copy link
Member

you are allowed to have only one uid/gid with systemd-homed. All you can do is to set ignore_chown_errors and squash all images to use one single ID

@kir68k
Copy link

kir68k commented Sep 20, 2023

you are allowed to have only one uid/gid with systemd-homed. All you can do is to set ignore_chown_errors and squash all images to use one single ID

So is this simply unsolvable for now, due to the way both of these programs work?

@IPlayZed
Copy link
Author

IPlayZed commented Sep 20, 2023 •
edited
Loading

you are allowed to have only one uid/gid with systemd-homed. All you can do is to set ignore_chown_errors and squash all images to use one single ID

I am a bit confused with this situation, the Systemd issue @mheon presented is closed and I would assume that this issue is fixed, but it seems like not? I do not even know which software (Podman or Homed) should be considered as breaking?

I would assume that distributions will adopt Homed in the future as it really has it's upsides, does that mean that Podman would break for all of those users?

@rhatdan
Copy link
Member

rhatdan commented Sep 22, 2023

In order to setup a user namespace, users have to have entries in /etc/subuid and /etc/subgid, If systemd-homed is not populating these filed with entries then users can not create UIDs different then their default UID, which breaks all containers. If you think this is a bug then it needs to be reported to systemd, there is nothing that the podman team can do.

@rhatdan rhatdan closed this as completed Sep 22, 2023
@containers containers locked and limited conversation to collaborators Sep 22, 2023
@rhatdan rhatdan converted this issue into discussion #20099 Sep 22, 2023

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@giuseppe @rhatdan @mheon @IPlayZed @kir68k