Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to import the required Python library (ansible[azure] (azure >= 2.0.0) #1463

Open
ericcames opened this issue Feb 23, 2024 · 24 comments
Labels
dependencies Pull requests that update a dependency file medium_priority Medium priority question Further information is requested work in In trying to solve, or in working with contributors

Comments

@ericcames
Copy link

SUMMARY

Trying to use this module azure.azcollection.azure_rm_account_info with ansible automation platform 2.3 and we are getting the failure list in the title

ISSUE TYPE
  • Bug Report
COMPONENT NAME

azure.azcollection.azure_rm_account_info

ANSIBLE VERSION
[awx@aap ~]$ ansible --version
ansible [core 2.14.11]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /var/lib/awx/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.16 (main, Sep 22 2023, 17:57:55) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] (/usr/bin/python3.9)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION
azure.azcollection
version: 1.18.1
and
version: latest
CONFIGURATION
[defaults]
action_warnings = false
callbacks_enabled = profile_tasks
show_custom_stats = true
# inject_facts_as_vars = no
deprecation_warnings = false
forks = 10
gathering = explicit
roles_path = ./roles/
collections_path = ./collections/

[privilege_escalation]
become = false
become_method = sudo
become_user = root

[persistent_connection]
command_timeout = 30
OS / ENVIRONMENT

azure subscription account
AAP 2.3 on RHEL 8

STEPS TO REPRODUCE

Set up a Microsoft Azure Resource Manager credential in AAP
Create a template to run the playbook listed below
Use this execution environment: quay.io/seanlee/ee-azure:1.12.0

---
- name: Get account info from Azure
  hosts: localhost
  connection: local

  tasks:

    - name: Get facts for current logged in user
      azure.azcollection.azure_rm_account_info:
...
EXPECTED RESULTS

I expect the playbook to pull back the account info.

ACTUAL RESULTS
{
  "exception": "Traceback (most recent call last):\n  File \"/tmp/ansible_azure.azcollection.azure_rm_account_info_payload_lev0wrek/ansible_azure.azcollection.azure_rm_account_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\", line 256, in <module>\n    from azure.storage.blob import BlobServiceClient\nImportError: cannot import name 'BlobServiceClient' from 'azure.storage.blob' (/usr/local/lib/python3.8/site-packages/azure/storage/blob/__init__.py)\n",
  "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on aee9dfd4ba17's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter",
  "invocation": {
    "module_args": {
      "auth_source": "auto",
      "cloud_environment": "AzureCloud",
      "api_profile": "latest",
      "profile": null,
      "subscription_id": null,
      "client_id": null,
      "secret": null,
      "tenant": null,
      "ad_user": null,
      "password": null,
      "cert_validation_mode": null,
      "adfs_authority_url": null,
      "log_mode": null,
      "log_path": null,
      "x509_certificate_path": null,
      "thumbprint": null
    }
  },
  "_ansible_no_log": false,
  "changed": false
}
@ericcames
Copy link
Author

@thisguyshouldworkforus I created an issue for what we are seeing.

@Fred-sun Fred-sun added medium_priority Medium priority work in In trying to solve, or in working with contributors labels Feb 26, 2024
@Fred-sun
Copy link
Collaborator

@ericcames After you installed aze.azcollection, did you install the related dependency files? Reinstall all dependent files and try again. Thank you!

@ericcames
Copy link
Author

@ericcames After you installed aze.azcollection, did you install the related dependency files? Reinstall all dependent files and try again. Thank you!

@Fred-sun thank you for the feedback. When I read the dependencies section of the documentation is says there are none. I am installing the collections via the collections directory and a requirements file.
image

Not sure if you are redhat employee or not if you are you can find me via [email protected]. I am working with a customer to prove out that this collection works in their environment. We are looking for ideas on how to get this collection working. Thanks for your feedback!

@A-Beck
Copy link

A-Beck commented Feb 26, 2024

I think the console is not loading dependencies correctly. You can see the python dependencies in the repo for the collection: https://github.com/ansible-collections/azure/blob/dev/requirements-azure.txt

Since you're using AAP, you should have access to the supported EE that includes these dependencies, IIRC

@skertz
Copy link

skertz commented Feb 26, 2024

Tried the requirements-azure.txt with python3.6 and 3.7 both report "msgraph-sdk==1.0.0 not found".
Using python3.8 and python 3.11 gets through this. However, running a playbook reports errors, ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'. Tried on fresh install of rh7 and rh8 machines.

@ericcames
Copy link
Author

@alexander-aps tagging

@Fred-sun
Copy link
Collaborator

Tried the requirements-azure.txt with python3.6 and 3.7 both report "msgraph-sdk==1.0.0 not found". Using python3.8 and python 3.11 gets through this. However, running a playbook reports errors, ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'. Tried on fresh install of rh7 and rh8 machines.

@skertz This error is strange because we did not ask to install 'azure-storage'. Could you please provide your detailed execution environment and error information? Thank you!

@Fred-sun
Copy link
Collaborator

@ericcames Can you provide 'pip3 freeze' information and python version information? Thank you!

@Fred-sun Fred-sun added the dependencies Pull requests that update a dependency file label Feb 27, 2024
@ericcames
Copy link
Author

@Fred-sun are you wanting this from the container or the controller?? This is the container we are using registry.redhat.io/ansible-automation-platform-23/ee-29-rhel8:latest

@ericcames
Copy link
Author

@Fred-sun we created quay.io/zigfreed/ee-azure:latest and have validated that it works as expected with three other asure collections. ---

  • name: Get info Azure Resource Group
    hosts: localhost
    connection: local

    tasks:

    • name: Get all resource Groups info
      register: resourcegroupinfo
      azure.azcollection.azure_rm_resourcegroup_info:

    • name: Print resourcegroupinfo
      ansible.builtin.debug:
      var: resourcegroupinfo
      ...


  • name: Get info Azure subscription_info
    hosts: localhost
    connection: local

    tasks:

    • name: Get facts for all subscriptions, including ones that are disabled.
      register: subscription_info
      azure.azcollection.azure_rm_subscription_info:
      all: true

    • name: Print subscription_info
      ansible.builtin.debug:
      var: subscription_info
      ...


  • name: Create Azure Resource Group
    hosts: localhost
    connection: local
    vars:
    az_resource_group_name: 'woopdidoo'
    location: 'westus'
    customer: 'mickey.mouse'

    tasks:

    • name: Create a resource group
      azure.azcollection.azure_rm_resourcegroup:
      name: "{{ az_resource_group_name }}"
      location: "{{ location }}"
      state: present
      tags:
      created: ansbile
      delete: never
      demo: "{{ customer }}"
      ...

@ericcames
Copy link
Author

This one does not work: ---

  • name: Get account info from Azure
    hosts: localhost
    connection: local

    tasks:

    • name: Get facts for current logged in user
      azure.azcollection.azure_rm_account_info:
      ...

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: error: MainError(additional_data={}, code='BadRequest', details=None, inner_error=InnerError(additional_data={'date': DateTime(2024, 2, 27, 12, 7, 25, tzinfo=Timezone('UTC'))}, client_request_id='24c090c3-8b51-4d10-bba2-d93ba2af6560', date=None, odata_type=None, request_id='f5d97fd3-a0cd-4f9e-8e69-02c50dd1c1b5'), message='/me request is only valid with delegated authentication flow.', target=None)
fatal: [localhost]: FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python3.9"}, "changed": false, "module_stderr": "Traceback (most recent call last):\n File "/home/runner/.ansible/tmp/ansible-tmp-1709035643.134935-26-217118459765709/AnsiballZ_azure_rm_account_info.py", line 107, in \n _ansiballz_main()\n File "/home/runner/.ansible/tmp/ansible-tmp-1709035643.134935-26-217118459765709/AnsiballZ_azure_rm_account_info.py", line 99, in _ansiballz_main\n …

@skertz
Copy link

skertz commented Feb 27, 2024

@Fred-sun

Redhat 8

ansible [core 2.15.3]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.11/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /bin/ansible
python version = 3.11.5 (main, Sep 22 2023, 15:34:29) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/usr/bin/python3.11)
jinja version = 3.1.2
libyaml = True

Install:

  1. ansible-galaxy collection install azure.azcollection
  2. python3 -m pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

(python3 is 3.6)
ERROR: Could not find a version that satisfies the requirement msgraph-sdk==1.0.0 (from versions: 1.0.0a0, 1.0.0a1, 1.0.0a2, 1.0.0a3, 1.0.0a4, 1.0.0a7, 1.0.0a8, 1.0.0a9, 1.0.0a10, 1.0.0a11, 1.0.0a12, 1.0.0a13)
ERROR: No matching distribution found for msgraph-sdk==1.0.0

  1. python3.11 -m pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

SUCCEEDS, but then:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'azure.storage.cloudstorageaccount'
fatal: [pd-ansible-3.production.iknowmed.com]: FAILED! => changed=false
msg: Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on pd-ansible-3's Python /usr/bin/python3.11. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter

Isn't azure-mgmt-storage and azure-storage-blob defined in the requirements?

@Fred-sun
Copy link
Collaborator

@skertz There are definitions, all of our requirements files are in ~/ requirements-aze.txt. Thank you!

packaging
requests[security]
xmltodict
msgraph-sdk==1.0.0
azure-cli-core==2.34.0
azure-common==1.1.11
azure-identity==1.14.0
azure-mgmt-authorization==2.0.0
azure-mgmt-apimanagement==3.0.0
azure-mgmt-batch==16.2.0
azure-mgmt-cdn==11.0.0
azure-mgmt-compute==26.1.0
azure-mgmt-containerinstance==9.0.0
azure-mgmt-core==1.3.0
azure-mgmt-containerregistry==9.1.0
azure-containerregistry==1.1.0
azure-mgmt-containerservice==20.0.0
azure-mgmt-datalake-store==1.0.0
azure-mgmt-datafactory==2.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-monitor==3.0.0
azure-mgmt-managedservices==6.0.0
azure-mgmt-managementgroups==1.0.0
azure-mgmt-network==19.1.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-privatedns==1.0.0
azure-mgmt-redis==13.0.0
azure-mgmt-resource==21.1.0
azure-mgmt-rdbms==10.0.0
azure-mgmt-search==8.0.0
azure-mgmt-servicebus==7.1.0
azure-mgmt-sql==3.0.1
azure-mgmt-storage==19.0.0
azure-mgmt-trafficmanager==1.0.0b1
azure-mgmt-web==6.1.0
azure-nspkg==2.0.0
azure-storage-blob==12.11.0
azure-core==1.28.0
azure-keyvault==4.2.0
azure-mgmt-keyvault==10.0.0
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-hdinsight==9.0.0
azure-mgmt-devtestlabs==9.0.0
azure-mgmt-loganalytics==12.0.0
azure-mgmt-automation==1.0.0
azure-mgmt-iothub==2.2.0
azure-iot-hub==2.6.1
azure-mgmt-recoveryservices==2.0.0
azure-mgmt-recoveryservicesbackup==3.0.0
azure-mgmt-notificationhubs==7.0.0
azure-mgmt-eventhub==10.1.0

@ericcames
Copy link
Author

ericcames commented Feb 29, 2024

@Fred-sun we built these into the execution environment we are using for testing.

quay.io/zigfreed/ee-azure:latest

@daviewales
Copy link

I'm getting this exact error trying to use azure.azcollection.azure_rm_keyvaultsecret_info.

I have installed using pipx as follows:

pipx install --include-deps ansible
pipx runpip ansible install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt

My ansible configuration:

❯ ansible --version
ansible [core 2.16.4]
  config file = /home/dwales/.ansible.cfg
  configured module search path = ['/home/dwales/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/dwales/.local/pipx/venvs/ansible/lib/python3.10/site-packages/ansible
  ansible collection location = /home/dwales/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/dwales/.local/bin/ansible
  python version = 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (/home/dwales/.local/pipx/venvs/ansible/bin/python)
  jinja version = 3.1.3
  libyaml = True

Error message:

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_azure.azcollection.azure_rm_keyvaultsecret_info_payload_x0pvcnz2/ansible_azure.azcollection.azure_rm_keyvaultsecret_info_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 240, in <module>
    from azure.mgmt.core.tools import parse_resource_id, resource_id, is_valid_resource_id
ModuleNotFoundError: No module named 'azure'
fatal: [1.2.3.4]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "auth_source": "auto",
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "log_mode": null,
            "log_path": null,
            "name": "secret",
            "password": null,
            "profile": null,
            "secret": null,
            "show_deleted_secret": false,
            "subscription_id": null,
            "tags": null,
            "tenant": null,
            "thumbprint": null,
            "vault_uri": "https://example.vault.azure.net",
            "version": "current",
            "x509_certificate_path": null
        }
    },
    "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on remote-vm's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

I think the problem is that it's trying to import from azure.mgmt.core.tools on the remote server, rather than the local machine where I have installed the requirements.txt. If I add delegate_to: 127.0.0.1, it works perfectly.

@Fred-sun
Copy link
Collaborator

Fred-sun commented Mar 7, 2024

@ericcames Sorry, there is an issue that has not been clarified, graphrbac migrated to msgraph, supported python version >=3.8. Please use aze.azcollection >=2.0.0 to run only in an environment higher than python3.8. Thank you!

@Fred-sun Fred-sun added the question Further information is requested label Mar 7, 2024
@ericcames
Copy link
Author

@Fred-sun Thank you!

@Fred-sun
Copy link
Collaborator

@ericcames Are you trying the new version? Has the problem been solved?

@nimbusparis
Copy link

I got the same issue and after many try/error, I succeed after manually install python3, ansible on the target computer. Then, I also installed the azure collection and run pip3 install .../requirements/txt all manipulations had been done with root

@Fred-sun
Copy link
Collaborator

@nimbusparis What does your error message look like? Can you provide a detailed log? Thank you!

@nimbusparis
Copy link

@nimbusparis What does your error message look like? Can you provide a detailed log? Thank you!

Well, now, I don't have the issue anymore, I can try to uninstall the collection and python module, but I hope it wouldn't break anything! 😆
Also, I can try to install the required collection/python module on the fly in my playbook.
In fact, I got the exact same error log as @ericcames at the begining of this thread.

@nimbusparis
Copy link

Here the vvvv log, hops it could help.

<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'echo ~user && sleep 0'"'"''
<51.255.204.38> (0, b'/home/user\n', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp `"&& mkdir "` echo /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822 `" && echo ansible-tmp-1715760217.3001661-69163-31083769795822="` echo /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822 `" ) && sleep 0'"'"''
<51.255.204.38> (0, b'ansible-tmp-1715760217.3001661-69163-31083769795822=/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822\n', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
Using module file /usr/lib/python3/dist-packages/ansible_collections/azure/azcollection/plugins/modules/azure_rm_storageblob.py
<51.255.204.38> PUT /root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd TO /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py
<51.255.204.38> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' '[51.255.204.38]'
<51.255.204.38> (0, b'sftp> put /root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py\n', b'OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/root/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/root/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master at \'/root/.ansible/cp/9afa1bc5be\'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 2\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug2: Server supports extension "[email protected]" revision 1\r\ndebug3: Sent message [email protected] I:1\r\ndebug3: Received limits reply T:201 I:1\r\ndebug3: server upload/download buffer sizes 261120 / 261120; using 261120 / 261120\r\ndebug3: server handle limit 1019; using 64\r\ndebug2: Sending SSH2_FXP_REALPATH "."\r\ndebug3: Sent message fd 3 T:16 I:2\r\ndebug3: SSH2_FXP_REALPATH . -> /home/user\r\ndebug3: Looking up /root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd\r\ndebug2: Sending SSH2_FXP_STAT "/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py"\r\ndebug3: Sent message fd 3 T:17 I:3\r\ndebug1: stat remote: No such file or directory\r\ndebug2: sftp_upload: upload local "/root/.ansible/tmp/ansible-local-69127aj5xlmin/tmp1a6bemrd" to remote "/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py"\r\ndebug2: Sending SSH2_FXP_OPEN "/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py"\r\ndebug3: Sent dest message SSH2_FXP_OPEN I:4 P:/home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py M:0x001a\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:0 S:154044\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 154044 bytes at 0\r\ndebug3: Sent message SSH2_FXP_CLOSE I:5\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'chmod u+x /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/ /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py && sleep 0'"'"''
<51.255.204.38> (0, b'', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' -tt 51.255.204.38 '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=mnwrxfhewwhoddyzirtxjtldezxfwwhl] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-mnwrxfhewwhoddyzirtxjtldezxfwwhl ; /usr/bin/python3 /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/AnsiballZ_azure_rm_storageblob.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<51.255.204.38> (1, b'\r\n\r\n{"exception": "Traceback (most recent call last):\\n  File \\"/tmp/ansible_azure.azcollection.azure_rm_storageblob_payload_1q1m5w5f/ansible_azure.azcollection.azure_rm_storageblob_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\\", line 223, in <module>\\n    from msrestazure.azure_active_directory import AADTokenCredentials\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrestazure/__init__.py\\", line 28, in <module>\\n    from .azure_configuration import AzureConfiguration\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrestazure/azure_configuration.py\\", line 34, in <module>\\n    from msrest import Configuration\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/__init__.py\\", line 28, in <module>\\n    from .configuration import Configuration\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/configuration.py\\", line 38, in <module>\\n    from .universal_http.requests import (\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/universal_http/__init__.py\\", line 53, in <module>\\n    from ..exceptions import ClientRequestError, raise_with_traceback\\n  File \\"/usr/local/lib/python3.10/dist-packages/msrest/exceptions.py\\", line 31, in <module>\\n    from azure.core.exceptions import SerializationError, DeserializationError\\nModuleNotFoundError: No module named \'azure.core\'\\n", "failed": true, "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on vps350593\'s Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter", "invocation": {"module_args": {"resource_group": "soft-rg", "storage_account_name": "softsa", "container": "artifacts", "blob": "soft-backend/soft-backend.zip", "dest": "/root/tmp/soft/backend.zip", "auth_source": "auto", "cloud_environment": "AzureCloud", "api_profile": "latest", "append_tags": true, "blob_type": "block", "force": false, "state": "present", "profile": null, "subscription_id": null, "client_id": null, "secret": null, "tenant": null, "ad_user": null, "password": null, "cert_validation_mode": null, "adfs_authority_url": null, "log_mode": null, "log_path": null, "x509_certificate_path": null, "thumbprint": null, "tags": null, "src": null, "batch_upload_src": null, "batch_upload_dst": null, "public_access": null, "content_type": null, "content_encoding": null, "content_language": null, "content_disposition": null, "cache_control": null, "content_md5": null}}}\r\n', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to 51.255.204.38 closed.\r\n")
<51.255.204.38> Failed to connect to the host via ssh: OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname 51.255.204.38 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 69017
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet_timeout: read header failed: Broken pipe
debug2: Received exit status from master 1
Shared connection to 51.255.204.38 closed.
<51.255.204.38> ESTABLISH SSH CONNECTION FOR USER: user
<51.255.204.38> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="user"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/9afa1bc5be"' 51.255.204.38 '/bin/sh -c '"'"'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1715760217.3001661-69163-31083769795822/ > /dev/null 2>&1 && sleep 0'"'"''
<51.255.204.38> (0, b'', b"OpenSSH_9.6p1 Ubuntu-3ubuntu13, OpenSSL 3.0.13 30 Jan 2024\r\ndebug1: Reading configuration data /root/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 51.255.204.38 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master at '/root/.ansible/cp/9afa1bc5be'\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 69017\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet_timeout: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_azure.azcollection.azure_rm_storageblob_payload_1q1m5w5f/ansible_azure.azcollection.azure_rm_storageblob_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 223, in <module>
    from msrestazure.azure_active_directory import AADTokenCredentials
  File "/usr/local/lib/python3.10/dist-packages/msrestazure/__init__.py", line 28, in <module>
    from .azure_configuration import AzureConfiguration
  File "/usr/local/lib/python3.10/dist-packages/msrestazure/azure_configuration.py", line 34, in <module>
    from msrest import Configuration
  File "/usr/local/lib/python3.10/dist-packages/msrest/__init__.py", line 28, in <module>
    from .configuration import Configuration
  File "/usr/local/lib/python3.10/dist-packages/msrest/configuration.py", line 38, in <module>
    from .universal_http.requests import (
  File "/usr/local/lib/python3.10/dist-packages/msrest/universal_http/__init__.py", line 53, in <module>
    from ..exceptions import ClientRequestError, raise_with_traceback
  File "/usr/local/lib/python3.10/dist-packages/msrest/exceptions.py", line 31, in <module>
    from azure.core.exceptions import SerializationError, DeserializationError
ModuleNotFoundError: No module named 'azure.core'
fatal: [51.255.204.38]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": "auto",
            "batch_upload_dst": null,
            "batch_upload_src": null,
            "blob": "soft-backend/soft-backend.zip",
            "blob_type": "block",
            "cache_control": null,
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "container": "artifacts",
            "content_disposition": null,
            "content_encoding": null,
            "content_language": null,
            "content_md5": null,
            "content_type": null,
            "dest": "/root/tmp/soft/backend.zip",
            "force": false,
            "log_mode": null,
            "log_path": null,
            "password": null,
            "profile": null,
            "public_access": null,
            "resource_group": "soft-rg",
            "secret": null,
            "src": null,
            "state": "present",
            "storage_account_name": "softsa",
            "subscription_id": null,
            "tags": null,
            "tenant": null,
            "thumbprint": null,
            "x509_certificate_path": null
        }
    },
    "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on vps350593's Python /usr/bin/python3. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"
}

@Fred-sun
Copy link
Collaborator

@nimbusparis Can you provide your test script? I'd like to try, thank you very much!

@nimbusparis
Copy link

Inventory:

all:
  children:
    ovh:
      hosts: xxx.xxx.xxx.xxx

group_vars/ovh.yaml

servicePrincipal:
  appId: xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx
  secret: fdsdfdsfsdfsdfsdfsf
  tenant:  xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx

Playbook:

- name: Deployment
  remote_user: user
  become: True
  hosts: ovh

  tasks:
  - name: Install Python3
    ansible.builtin.raw: apt install python3
       
  - name: Login to Azure
    # Use Azure CLI (installed manually) to connect to Azure
    ansible.builtin.command: "az login --service-principal -u {{servicePrincipal.appId}} -p {{servicePrincipal.secret}} -t {{servicePrincipal.tenant}}" 

  - name: Download Pixhint packages
    azure.azcollection.azure_rm_storageblob:
      resource_group: soft-rg
      storage_account_name: softsa
      container: artifacts
      blob: "soft/soft-backend.zip"
      dest: "~/tmp/soft/soft-backend.zip"
     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file medium_priority Medium priority question Further information is requested work in In trying to solve, or in working with contributors
Projects
None yet
Development

No branches or pull requests

6 participants