Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to access remote podman containers via intermediate ssh host #246

Open
gmarcy opened this issue Apr 18, 2021 · 6 comments
Open

Unable to access remote podman containers via intermediate ssh host #246

gmarcy opened this issue Apr 18, 2021 · 6 comments

Comments

@gmarcy
Copy link

gmarcy commented Apr 18, 2021

/kind bug

Description

I have a playbook to create a podman container on either a remote or a local machine. The podman_container task creates the container and runs it fine in either case. After I add the container to the hosts group and try to use the podman connection to reach it the local play works but the remote play fails.

Steps to reproduce the issue:

  1. Run playbook using remote tag

Describe the results you received:

fatal: [remotehost]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory.In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in "/tmp", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p "echo ~/.ansible/tmp"&& mkdir "echo ~/.ansible/tmp/ansible-tmp-1618771081.7663367-103788-258354183528348" && echo ansible-tmp-1618771081.7663367-103788-258354183528348="echo ~/.ansible/tmp/ansible-tmp-1618771081.7663367-103788-258354183528348" ), exited with result 125", "unreachable": true}

Describe the results you expected:

Run playbook using local tag and it works. Would like to be able to use the connector on remote machines and not just on the ansible install machine.

Additional information you deem important (e.g. issue happens only occasionally):

I tried several alternatives found with google searches, including several variations of adding
ansible_ssh_host: remotehost but none were successful.

Version of the containers.podman collection:
Either git commit if installed from git: git show --summary
Or version from ansible-galaxy if installed from galaxy: ansible-galaxy collection list | grep containers.podman

Installing 'containers.podman:1.5.0' to '/home/gmarcy/.ansible/collections/ansible_collections/containers/podman'

Output of ansible --version:

ansible 2.9.18

Output of podman version:

Version:      3.0.0-dev

Output of podman info --debug:

(paste your output here)

Package info (e.g. output of rpm -q podman or apt list podman):

(paste your output here)

Inventory file (e.g. content of inv):

[targets]
localhost   ansible_connection=local  ansible_python_interpreter=/usr/bin/python3
remotehost  ansible_connection=ssh    ansible_user=root

Playbook you run with ansible (e.g. content of playbook.yaml):

- hosts: remotehost
  tags: remote
  tasks:
  - name: create remote podman container
    containers.podman.podman_container:
      name: remote_container
      image: registry.fedoraproject.org/fedora:33
      command: sleep infinity
      remove: yes
    register: remote_container_info
  - name: add remote container to hosts
    add_host:
      hostname: "{{ remote_container_info.container.Name }}"
      ansible_connection: containers.podman.podman
      ansible_python_interpreter: /usr/bin/python3
  - name: run dnf to bring remote container up to date
    delegate_to: remote_container
    dnf:
      state: latest

- hosts: localhost
  tags: local
  tasks:
  - name: create local podman container
    containers.podman.podman_container:
      name: local_container
      image: registry.fedoraproject.org/fedora:33
      command: sleep infinity
      remove: yes
    register: local_container_info
  - name: add local container to hosts
    add_host:
      hostname: "{{ local_container_info.container.Name }}"
      ansible_connection: containers.podman.podman
      ansible_python_interpreter: /usr/bin/python3
  - name: run dnf to bring local container up to date
    delegate_to: local_container
    dnf:
      state: latest

Command line and output of ansible run with high verbosity

ansible-playbook -vvvvvvvv -i inv ./playbook.yml -t remote

ansible-playbook 2.9.18
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/gmarcy/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.6/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 3.6.8 (default, Jan 27 2021, 01:17:18) [GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/gmarcy/ansible/inv as it did not pass its verify_file() method
script declined parsing /home/gmarcy/ansible/inv as it did not pass its verify_file() method
auto declined parsing /home/gmarcy/ansible/inv as it did not pass its verify_file() method
Set default localhost to localhost
Parsed /home/gmarcy/ansible/inv inventory source with ini plugin
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python3.6/site-packages/ansible/plugins/callback/default.py
Attempting to use 'actionable' callback.
Skipping callback 'actionable', as we already have a stdout callback.
Attempting to use 'aws_resource_actions' callback.

<snip/>

Attempting to use 'yaml' callback.
Skipping callback 'yaml', as we already have a stdout callback.

PLAYBOOK: playbook.yml *********************************************************************************************************************************************************************************
Positional arguments: ./playbook.yml
verbosity: 8
connection: smart
timeout: 10
become_method: sudo
tags: ('remote',)
inventory: ('/home/gmarcy/ansible/inv',)
forks: 5
2 plays in ./playbook.yml

PLAY [remotehost] **************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************
task path: /home/gmarcy/ansible/playbook.yml:1
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
<remotehost> (0, b'/root\n', b'OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n ... <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277 `" && echo ansible-tmp-1618771932.594448-104277-277618122373277="` echo /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277 `" ) && sleep 0'"'"''
<remotehost> (0, b'ansible-tmp-1618771932.594448-104277-277618122373277=/root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277\n', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/facts/namespace.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/facts/default_collectors.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/facts/ansible_collector.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/facts/__init__.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/basic.py

<snip/>

Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/common/parameters.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/compat/selectors.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/compat/__init__.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/common/collections.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/compat/_selectors2.py
<remotehost> Attempting python interpreter discovery
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.5'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''
<remotehost> (0, b'PLATFORM\nLinux\nFOUND\n/usr/bin/python3.6\n/usr/libexec/platform-python\n/usr/bin/python3\nENDFOUND\n', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'/usr/bin/python3.6 && sleep 0'"'"''
<remotehost> (0, b'{"platform_dist_result": ["centos", "8", ""], "osrelease_content": "NAME=\\"CentOS Stream\\"\\nVERSION=\\"8\\"\\nID=\\"centos\\"\\nID_LIKE=\\"rhel fedora\\"\\nVERSION_ID=\\"8\\"\\nPLATFORM_ID=\\"platform:el8\\"\\nPRETTY_NAME=\\"CentOS Stream 8\\"\\nANSI_COLOR=\\"0;31\\"\\nCPE_NAME=\\"cpe:/o:centos:centos:8\\"\\nHOME_URL=\\"https://centos.org/\\"\\nBUG_REPORT_URL=\\"https://bugzilla.redhat.com/\\"\\nREDHAT_SUPPORT_PRODUCT=\\"Red Hat Enterprise Linux 8\\"\\nREDHAT_SUPPORT_PRODUCT_VERSION=\\"CentOS Stream\\"\\n"}\n', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
Using module file /usr/lib/python3.6/site-packages/ansible/modules/system/setup.py
<remotehost> PUT /home/gmarcy/.ansible/tmp/ansible-local-104271igln08s5/tmpruoduti0 TO /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277/AnsiballZ_setup.py
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set sftp_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 '[remotehost]'
<remotehost> (0, b'sftp> put /home/gmarcy/.ansible/tmp/ansible-local-104271igln08s5/tmpruoduti0 /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277/AnsiballZ_setup.py\n', b'OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277/ /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277/AnsiballZ_setup.py && sleep 0'"'"''
<remotehost> (0, b'', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 -tt remotehost '/bin/sh -c '"'"'/usr/libexec/platform-python /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277/AnsiballZ_setup.py && sleep 0'"'"''
<remotehost> (0, b'\r\n{"ansible_facts": {"ansible_user_id": "root", "ansible_user_uid": 0, "ansible_user_gid": 0, "ansible_user_gecos": "root", "ansible_user_dir": "/root", "ansible_user_shell": "/bin/bash", "ansible_real_user_id": 0, "ansible_effective_user_id": 0, "ansible_real_group_id": 0, "ansible_effective_group_id": 0, "ansible_system_capabilities_enforced": "True", "ansible_system_capabilities": ["cap_chown", "cap_dac_override", "cap_dac_read_search", "cap_fowner", "cap_fsetid", "cap_kill", "cap_setgid", "cap_setuid", "cap_setpcap", "cap_linux_immutable", "cap_net_bind_service", "cap_net_broadcast", "cap_net_admin", "cap_net_raw", "cap_ipc_lock", "cap_ipc_owner", "cap_sys_module", "cap_sys_rawio", "cap_sys_chroot", "cap_sys_ptrace", "cap_sys_pacct", "cap_sys_admin", "cap_sys_boot", "cap_sys_nice", "cap_sys_resource", "cap_sys_time", "cap_sys_tty_config", "cap_mknod", "cap_lease", "cap_audit_write", "cap_audit_control", "cap_setfcap", "cap_mac_override", "cap_mac_admin", "cap_syslog", "cap_wake_alarm", "cap_block_suspend", "cap_audit_read", "38", "39+ep"], "ansible_apparmor": {"status": "disabled"}, "ansible_selinux_python_present": true, "ansible_selinux": {"status": "enabled", "policyvers": 33, "config_mode": "disabled", "mode": "permissive", "type": "targeted"}, "ansible_system": "Linux", "ansible_kernel": "4.18.0-294.el8.x86_64", "ansible_kernel_version": "#1 SMP Mon Mar 15 22:38:42 UTC 2021", "ansible_machine": "x86_64", "ansible_python_version": "3.6.8", <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1618771932.594448-104277-277618122373277/ > /dev/null 2>&1 && sleep 0'"'"''
<remotehost> (0, b'', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
ok: [remotehost]
META: ran handlers

TASK [create remote podman container] ******************************************************************************************************************************************************************
task path: /home/gmarcy/ansible/playbook.yml:4
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
<remotehost> (0, b'/root\n', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928 `" && echo ansible-tmp-1618771933.7978294-104291-64264670523928="` echo /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928 `" ) && sleep 0'"'"''
<remotehost> (0, b'ansible-tmp-1618771933.7978294-104291-64264670523928=/root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928\n', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
Using module_utils file ansible_collections/containers/podman/plugins/module_utils/podman
Using module_utils file ansible_collections/containers/podman/plugins/module_utils
Using module_utils file ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib
Using module_utils file ansible_collections/containers/podman
Using module_utils file ansible_collections
Using module_utils file ansible_collections/containers/podman/plugins

<snip/>

Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/distro/__init__.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/distro/_distro.py
Using module_utils file /usr/lib/python3.6/site-packages/ansible/module_utils/compat/_selectors2.py
Using module file /home/gmarcy/.ansible/collections/ansible_collections/containers/podman/plugins/modules/podman_container.py
<remotehost> PUT /home/gmarcy/.ansible/tmp/ansible-local-104271igln08s5/tmps9sts5ce TO /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928/AnsiballZ_podman_container.py
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set sftp_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 '[remotehost]'
<remotehost> (0, b'sftp> put /home/gmarcy/.ansible/tmp/ansible-local-104271igln08s5/tmps9sts5ce /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928/AnsiballZ_podman_container.py\n', b'OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928/ /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928/AnsiballZ_podman_container.py && sleep 0'"'"''
<remotehost> (0, b'', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 -tt remotehost '/bin/sh -c '"'"'/usr/libexec/platform-python /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928/AnsiballZ_podman_container.py && sleep 0'"'"''
<remotehost> (0, b'\r\n{"changed": true, "actions": ["started remote_container"], "container": {"Id": "35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4", "Created": "2021-04-18T14:52:14.497637562-04:00", "Path": "sleep", "Args": ["infinity"], "State": {"OciVersion": "1.0.2-dev", "Status": "running", "Running": true, "Paused": false, "Restarting": false, "OOMKilled": false, "Dead": false, "Pid": 77732, "ConmonPid": 77729, "ExitCode": 0, "Error": "", "StartedAt": "2021-04-18T14:52:14.737126612-04:00", "FinishedAt": "0001-01-01T00:00:00Z", "Healthcheck": {"Status": "", "FailingStreak": 0, "Log": null}},  <snip/>
<remotehost> ESTABLISH SSH CONNECTION FOR USER: root
<remotehost> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<remotehost> SSH: ansible_password/ansible_ssh_password not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<remotehost> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User="root")
<remotehost> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=10)
<remotehost> SSH: PlayContext set ssh_common_args: ()
<remotehost> SSH: PlayContext set ssh_extra_args: ()
<remotehost> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/home/gmarcy/.ansible/cp/7379176439)
<remotehost> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/gmarcy/.ansible/cp/7379176439 remotehost '/bin/sh -c '"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1618771933.7978294-104291-64264670523928/ > /dev/null 2>&1 && sleep 0'"'"''
<remotehost> (0, b'', b"OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020\r\n <snip/>
changed: [remotehost] => {
    "actions": [
        "started remote_container"
    ],
    "changed": true,
    "container": {
        "AppArmorProfile": "",
        "Args": [
            "infinity"
        ],
        "BoundingCaps": [
            "CAP_NET_RAW",
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FOWNER",
            "CAP_FSETID",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE",
            "CAP_SETFCAP",
            "CAP_SETGID",
            "CAP_SETPCAP",
            "CAP_SETUID",
            "CAP_SYS_CHROOT"
        ],
        "Config": {
            "Annotations": {
                "io.container.manager": "libpod",
                "io.kubernetes.cri-o.Created": "2021-04-18T14:52:14.497637562-04:00",
                "io.kubernetes.cri-o.TTY": "false",
                "io.podman.annotations.autoremove": "TRUE",
                "io.podman.annotations.init": "FALSE",
                "io.podman.annotations.privileged": "FALSE",
                "io.podman.annotations.publish-all": "FALSE",
                "org.opencontainers.image.stopSignal": "15"
            },
            "AttachStderr": false,
            "AttachStdin": false,
            "AttachStdout": false,
            "Cmd": [
                "sleep",
                "infinity"
            ],
            "CreateCommand": [
                "podman",
                "container",
                "run",
                "--name",
                "remote_container",
                "--rm",
                "--detach=True",
                "registry.fedoraproject.org/fedora:33",
                "sleep",
                "infinity"
            ],
            "Domainname": "",
            "Entrypoint": "",
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "TERM=xterm",
                "container=oci",
                "DISTTAG=f33container",
                "FGC=f33",
                "HOME=/root",
                "HOSTNAME=35c54928aec9"
            ],
            "Hostname": "35c54928aec9",
            "Image": "registry.fedoraproject.org/fedora:33",
            "Labels": {
                "license": "MIT",
                "name": "fedora",
                "vendor": "Fedora Project",
                "version": "33"
            },
            "OnBuild": null,
            "OpenStdin": false,
            "StdinOnce": false,
            "StopSignal": 15,
            "Tty": false,
            "Umask": "0022",
            "User": "",
            "Volumes": null,
            "WorkingDir": "/"
        },
        "ConmonPidFile": "/run/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata/conmon.pid",
        "Created": "2021-04-18T14:52:14.497637562-04:00",
        "Dependencies": [],
        "Driver": "overlay",
        "EffectiveCaps": [
            "CAP_NET_RAW",
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FOWNER",
            "CAP_FSETID",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE",
            "CAP_SETFCAP",
            "CAP_SETGID",
            "CAP_SETPCAP",
            "CAP_SETUID",
            "CAP_SYS_CHROOT"
        ],
        "ExecIDs": [],
        "ExitCommand": [
            "/usr/bin/podman",
            "--root",
            "/var/lib/containers/storage",
            "--runroot",
            "/run/containers/storage",
            "--log-level",
            "warning",
            "--cgroup-manager",
            "systemd",
            "--tmpdir",
            "/run/libpod",
            "--runtime",
            "crun",
            "--storage-driver",
            "overlay",
            "--storage-opt",
            "overlay.mountopt=nodev,metacopy=on",
            "--events-backend",
            "file",
            "container",
            "cleanup",
            "--rm",
            "35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4"
        ],
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/containers/storage/overlay/d9e1d1e08de26f234a83c6c737827884dd15c68c80714a5a973d245ed456f7a1/diff",
                "MergedDir": "/var/lib/containers/storage/overlay/71320a8d4f327895761e32d58ede5273c788ea93854f4350c7d8d1f1f2e7c707/merged",
                "UpperDir": "/var/lib/containers/storage/overlay/71320a8d4f327895761e32d58ede5273c788ea93854f4350c7d8d1f1f2e7c707/diff",
                "WorkDir": "/var/lib/containers/storage/overlay/71320a8d4f327895761e32d58ede5273c788ea93854f4350c7d8d1f1f2e7c707/work"
            },
            "Name": "overlay"
        },
        "HostConfig": {
            "AutoRemove": true,
            "Binds": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceWriteIOps": null,
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "CapAdd": [],
            "CapDrop": [
                "CAP_AUDIT_WRITE",
                "CAP_MKNOD"
            ],
            "Cgroup": "",
            "CgroupConf": null,
            "CgroupManager": "systemd",
            "CgroupMode": "host",
            "CgroupParent": "",
            "Cgroups": "default",
            "ConsoleSize": [
                0,
                0
            ],
            "ContainerIDFile": "",
            "CpuCount": 0,
            "CpuPercent": 0,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpuShares": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": [],
            "GroupAdd": [],
            "IOMaximumBandwidth": 0,
            "IOMaximumIOps": 0,
            "IpcMode": "private",
            "Isolation": "",
            "KernelMemory": 0,
            "Links": null,
            "LogConfig": {
                "Config": null,
                "Path": "/var/lib/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata/ctr.log",
                "Size": "0B",
                "Tag": "",
                "Type": "k8s-file"
            },
            "Memory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": 0,
            "NanoCpus": 0,
            "NetworkMode": "bridge",
            "OomKillDisable": false,
            "OomScoreAdj": 0,
            "PidMode": "private",
            "PidsLimit": 2048,
            "PortBindings": {},
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "RestartPolicy": {
                "MaximumRetryCount": 0,
                "Name": ""
            },
            "Runtime": "oci",
            "SecurityOpt": [],
            "ShmSize": 65536000,
            "Tmpfs": {},
            "UTSMode": "private",
            "Ulimits": [
                {
                    "Hard": 1048576,
                    "Name": "RLIMIT_NOFILE",
                    "Soft": 1048576
                },
                {
                    "Hard": 4194304,
                    "Name": "RLIMIT_NPROC",
                    "Soft": 4194304
                }
            ],
            "UsernsMode": "",
            "VolumeDriver": "",
            "VolumesFrom": null
        },
        "HostnamePath": "/run/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata/hostname",
        "HostsPath": "/run/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata/hosts",
        "Id": "35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4",
        "Image": "9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d",
        "ImageName": "registry.fedoraproject.org/fedora:33",
        "IsInfra": false,
        "MountLabel": "system_u:object_r:container_file_t:s0:c399,c1014",
        "Mounts": [],
        "Name": "remote_container",
        "Namespace": "",
        "NetworkSettings": {
            "Bridge": "",
            "EndpointID": "",
            "Gateway": "10.88.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "HairpinMode": false,
            "IPAddress": "10.88.0.13",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "MacAddress": "46:52:0a:ed:c8:1a",
            "Networks": {
                "podman": {
                    "DriverOpts": null,
                    "EndpointID": "",
                    "Gateway": "10.88.0.1",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "IPAMConfig": null,
                    "IPAddress": "10.88.0.13",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "Links": null,
                    "MacAddress": "46:52:0a:ed:c8:1a",
                    "NetworkID": "podman"
                }
            },
            "Ports": {},
            "SandboxID": "",
            "SandboxKey": "/run/netns/cni-a7963dc6-bb64-d06c-7de2-6475883cbe04"
        },
        "OCIConfigPath": "/var/lib/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata/config.json",
        "OCIRuntime": "crun",
        "Path": "sleep",
        "Pod": "",
        "ProcessLabel": "system_u:system_r:container_t:s0:c399,c1014",
        "ResolvConfPath": "/run/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata/resolv.conf",
        "RestartCount": 0,
        "Rootfs": "",
        "State": {
            "ConmonPid": 77729,
            "Dead": false,
            "Error": "",
            "ExitCode": 0,
            "FinishedAt": "0001-01-01T00:00:00Z",
            "Healthcheck": {
                "FailingStreak": 0,
                "Log": null,
                "Status": ""
            },
            "OOMKilled": false,
            "OciVersion": "1.0.2-dev",
            "Paused": false,
            "Pid": 77732,
            "Restarting": false,
            "Running": true,
            "StartedAt": "2021-04-18T14:52:14.737126612-04:00",
            "Status": "running"
        },
        "StaticDir": "/var/lib/containers/storage/overlay-containers/35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4/userdata"
    },
    "invocation": {
        "module_args": {
            "annotation": null,
            "authfile": null,
            "blkio_weight": null,
            "blkio_weight_device": null,
            "cap_add": null,
            "cap_drop": null,
            "cgroup_parent": null,
            "cgroupns": null,
            "cgroups": null,
            "cidfile": null,
            "cmd_args": null,
            "command": "sleep infinity",
            "conmon_pidfile": null,
            "cpu_period": null,
            "cpu_rt_period": null,
            "cpu_rt_runtime": null,
            "cpu_shares": null,
            "cpus": null,
            "cpuset_cpus": null,
            "cpuset_mems": null,
            "debug": false,
            "detach": true,
            "detach_keys": null,
            "device": null,
            "device_read_bps": null,
            "device_read_iops": null,
            "device_write_bps": null,
            "device_write_iops": null,
            "dns": null,
            "dns_option": null,
            "dns_search": null,
            "entrypoint": null,
            "env": null,
            "env_file": null,
            "env_host": null,
            "etc_hosts": null,
            "executable": "podman",
            "expose": null,
            "force_restart": false,
            "gidmap": null,
            "group_add": null,
            "healthcheck": null,
            "healthcheck_interval": null,
            "healthcheck_retries": null,
            "healthcheck_start_period": null,
            "healthcheck_timeout": null,
            "hostname": null,
            "http_proxy": null,
            "image": "registry.fedoraproject.org/fedora:33",
            "image_strict": false,
            "image_volume": null,
            "init": null,
            "init_path": null,
            "interactive": null,
            "ip": null,
            "ipc": null,
            "kernel_memory": null,
            "label": null,
            "label_file": null,
            "log_driver": null,
            "log_level": null,
            "log_opt": null,
            "mac_address": null,
            "memory": null,
            "memory_reservation": null,
            "memory_swap": null,
            "memory_swappiness": null,
            "mount": null,
            "name": "remote_container",
            "network": null,
            "no_hosts": null,
            "oom_kill_disable": null,
            "oom_score_adj": null,
            "pid": null,
            "pids_limit": null,
            "pod": null,
            "privileged": null,
            "publish": null,
            "publish_all": null,
            "read_only": null,
            "read_only_tmpfs": null,
            "recreate": false,
            "remove": true,
            "restart_policy": null,
            "rm": true,
            "rootfs": null,
            "security_opt": null,
            "shm_size": null,
            "sig_proxy": null,
            "state": "started",
            "stop_signal": null,
            "stop_timeout": null,
            "subgidname": null,
            "subuidname": null,
            "sysctl": null,
            "systemd": null,
            "tmpfs": null,
            "tty": null,
            "uidmap": null,
            "ulimit": null,
            "user": null,
            "userns": null,
            "uts": null,
            "volume": null,
            "volumes_from": null,
            "workdir": null
        }
    },
    "podman_actions": [
        "podman run --name remote_container --rm --detach=True registry.fedoraproject.org/fedora:33 sleep infinity"
    ],
    "stderr": "",
    "stderr_lines": [],
    "stdout": "35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4\n",
    "stdout_lines": [
        "35c54928aec9449ff51842a8b73b7c6b9dfd23c5b0249bdfa0832c3aacc55cc4"
    ]
}

TASK [add remote container to hosts] *******************************************************************************************************************************************************************
task path: /home/gmarcy/ansible/playbook.yml:11
creating host via 'add_host': hostname=remote_container
changed: [remotehost] => {
    "add_host": {
        "groups": [],
        "host_name": "remote_container",
        "host_vars": {
            "ansible_connection": "containers.podman.podman",
            "ansible_python_interpreter": "/usr/bin/python3"
        }
    },
    "changed": true
}

TASK [run dnf to bring remote container up to date] ****************************************************************************************************************************************************
task path: /home/gmarcy/ansible/playbook.yml:16
Using podman connection from collection
<remote_container> RUN [b'/usr/bin/podman', b'mount', b'remote_container']
STDOUT b''
STDERR b'Error: cannot run command "podman mount" in rootless mode, must execute `podman unshare` first\n'
RC CODE 125
Failed to mount container remote_container: b'Error: cannot run command "podman mount" in rootless mode, must execute `podman unshare` first'
<remote_container> RUN [b'/usr/bin/podman', b'exec', b'remote_container', b'/bin/sh', b'-c', b'echo ~ && sleep 0']
STDOUT b''
STDERR b'Error: no container with name or ID remote_container found: no such container\n'
RC CODE 125
STDOUT b'Error: no container with name or ID remote_container found: no such container\n' STDERR b'Error: no container with name or ID remote_container found: no such container\n'
<remote_container> RUN [b'/usr/bin/podman', b'exec', b'remote_container', b'/bin/sh', b'-c', b'echo "`pwd`" && sleep 0']
STDOUT b''
STDERR b'Error: no container with name or ID remote_container found: no such container\n'
RC CODE 125
STDOUT b'Error: no container with name or ID remote_container found: no such container\n' STDERR b'Error: no container with name or ID remote_container found: no such container\n'
<remote_container> RUN [b'/usr/bin/podman', b'exec', b'remote_container', b'/bin/sh', b'-c', b'( umask 77 && mkdir -p "` echo ~/.ansible/tmp `"&& mkdir "` echo ~/.ansible/tmp/ansible-tmp-1618771935.1186886-104302-114922103186874 `" && echo ansible-tmp-1618771935.1186886-104302-114922103186874="` echo ~/.ansible/tmp/ansible-tmp-1618771935.1186886-104302-114922103186874 `" ) && sleep 0']
STDOUT b''
STDERR b'Error: no container with name or ID remote_container found: no such container\n'
RC CODE 125
STDOUT b'Error: no container with name or ID remote_container found: no such container\n' STDERR b'Error: no container with name or ID remote_container found: no such container\n'
fatal: [remotehost]: UNREACHABLE! => {
    "changed": false,
    "msg": "Failed to create temporary directory.In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1618771935.1186886-104302-114922103186874 `\" && echo ansible-tmp-1618771935.1186886-104302-114922103186874=\"` echo ~/.ansible/tmp/ansible-tmp-1618771935.1186886-104302-114922103186874 `\" ), exited with result 125, stderr output: Error: no container with name or ID remote_container found: no such container\n",
    "unreachable": true
}

PLAY RECAP *********************************************************************************************************************************************************************************************
remotehost                 : ok=3    changed=2    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   

Additional environment details (AWS, VirtualBox, physical, etc.):

@gmarcy
Copy link
Author

gmarcy commented Apr 18, 2021

@TomasTomecek any thoughts/suggestions? Not sure if this is a configuration issue, or simply beyond the capabilities of non-ssh connection modules.

@sshnaidm
Copy link
Member

Why not to run podman connection remotely as well?
Anyway, probably you need to use podman-remote for that:
ansible_podman_executable: podman-remote

podman_executable:
description:
- Executable for podman command.
default: podman
vars:
- name: ansible_podman_executable
env:
- name: ANSIBLE_PODMAN_EXECUTABLE

At least it was designed for that. But podman-remote didn't support remote cp operation for a long time, then it changed a lot the way it works. So I doubt it will work now, probably need to figure out a new way to work with latest version of podman-remote.

@gmarcy
Copy link
Author

gmarcy commented Apr 22, 2021

Thanks @sshnaidm for the response.

I tried changing ansible_podman_executable to podman-remote and the output was:

Using podman connection from collection
<remote_container> RUN [b'/usr/bin/podman-remote', b'mount', b'remote_container']
STDOUT b''
STDERR b"Error: unrecognized command `podman-remote mount`\nTry 'podman-remote --help' for more information.\n"
RC CODE 125
Failed to mount container remote_container: b"Error: unrecognized command `podman-remote mount`\nTry 'podman-remote --help' for more information."
<remote_container> RUN [b'/usr/bin/podman-remote', b'exec', b'--user', b'root', b'remote_container', b'/bin/sh', b'-c', b'echo ~root && sleep 0']
STDOUT b''
STDERR b'Error: cannot connect to the Podman socket, please verify that Podman REST API service is running: Get "http://d/v3.1.0-dev/libpod/_ping": dial unix ///run/user/1000/podman/podman.sock: connect: no such file or directory\n'
RC CODE 125

Any ideas on what to attempt next? I am not sure how the podman-remote would be able to determine where the remote_container was created since that was done by a containers.podman.podman_container ansible task.
Are there some other podman commands I need to run to register that container with podman-remote?

@gmarcy
Copy link
Author

gmarcy commented Apr 23, 2021

After much additional trial and error I managed to get something to work. Still assessing how stable it is. One thing I did notice is that podman.service on the remote host is filling up with lots of conmon processes, several for each remotely executed command. They all appear to end with --exit-delay 300 so I'm guessing they will eventually go away, but would be nice if there was a way to be more proactive in cleaning them up.

Here is the latest version of the remotehost play in my playbook:

- hosts: remotehost
  tags: remote
  tasks:
  - name: Ensure user specific systemd instance are persistent
    command: |
      loginctl enable-linger {{ ansible_user_id }}
    register: systemd_instance_persist
    changed_when: "systemd_instance_persist.rc == 0"
  - name: Retrieve remote user runtime path
    command: |
      loginctl show-user {{ ansible_user_id }} -p RuntimePath --value
    register: systemd_runtime_path
  - name: Enable and start podman.socket
    systemd:
      name: podman.socket
      enabled: yes
      state: started
      scope: user
  - name: Start podman.service
    systemd:
      name: podman.service
      state: started
      scope: user
  - name: create remote podman container
    containers.podman.podman_container:
      name: remote_container
      image: registry.fedoraproject.org/fedora:33
      command: sleep infinity
  - name: Add remote system connection definition for remote_container
    command: |
      podman --remote system connection add remote_container --identity "{{ ansible_user_dir }}/.ssh/id_rsa" "ssh://{{ ansible_host }}{{ systemd_runtime_path.stdout }}/podman/podman.sock"
    delegate_to: localhost
  - name: add remote container to hosts
    add_host:
      hostname: remote_container
      ansible_connection: containers.podman.podman
      ansible_python_interpreter: /usr/bin/python3
      ansible_podman_extra_args: --remote
  - name: get container uname info
    command: |
      uname -a
    delegate_to: remote_container
  - name: run dnf to bring remote container up to date
    dnf:
      state: latest
    delegate_to: remote_container

Any suggestions on how to simplify or otherwise wrangle that unwieldy process appreciated.

@gmarcy
Copy link
Author

gmarcy commented Apr 23, 2021

Also, since there doesn't appear to be a buildah --remote option, is there any way to get a similar approach to work with the buildah connection plugin?

@gmarcy
Copy link
Author

gmarcy commented May 14, 2021

@sshnaidm @TomasTomecek any additional thoughts on being able to use buildah connector remotely? have a similar playbook with local and remote tags. the localhost version works great, but unable to get the remote equivalent to function.

- hosts: localhost
  tags: local
  tasks:
  - name: create local buildah container
    command: |
      buildah from --name local_buildah registry.fedoraproject.org/fedora:33
  - name: add local buildah container to hosts
    add_host:
      hostname: local_buildah
      ansible_connection: containers.podman.buildah
      ansible_python_interpreter: /usr/bin/python3
  - name: run dnf to bring local container up to date
    dnf:
      state: latest
    delegate_to: local_buildah
  - name: create the entrypoint script
    copy:
      content: |
        #!/bin/bash
        set -eo pipefail
        echo sleeping forever
        sleep infinity
      dest: /entrypoint.sh
      mode: 0755
    delegate_to: local_buildah
  - name: set entrypoint
    command: buildah config --entrypoint '["/entrypoint.sh"]' local_buildah
  - name: commit local buildah container
    command: buildah commit local_buildah local_image:latest

- hosts: remotehost
  tags: remote
  tasks:
  - name: create remote buildah container
    command: |
      buildah from --name remote_buildah registry.fedoraproject.org/fedora:33
  - name: add remote buildah container to hosts
    add_host:
      hostname: remote_buildah
      ansible_connection: containers.podman.buildah
      ansible_python_interpreter: /usr/bin/python3
  - name: run dnf to bring remote container up to date
    dnf:
      state: latest
    delegate_to: remote_buildah
  - name: create the entrypoint script
    copy:
      content: |
        #!/bin/bash
        set -eo pipefail
        echo sleeping forever
        sleep infinity
      dest: /entrypoint.sh
      mode: 0755
    delegate_to: remote_buildah
  - name: set entrypoint
    command: buildah config --entrypoint '["/entrypoint.sh"]' remote_buildah
  - name: commit remote buildah container
    command: buildah commit remote_buildah remote_image:latest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants