ansible-builder 3.1.0 encountering "ERROR: Double requirement given: dnspython"

[jasondickerson]

ISSUE TYPE

• Bug Report

SUMMARY

Encountering Issue with ansible-builder building an execution environment:

Ignoring ipaddress: markers 'python_version < "3.3"' don't match your environment
ERROR: Double requirement given: dnspython (from -r /tmp/src/requirements.txt (line 74)) (already in dnspython>=2.0.0 (from -r /tmp/src/requirements.txt (line 63)), name='dnspython')
Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1

Issue occurs whether dnspython is in the dependencies python list or not.
Environment is:

• Fedora 40
• ansible-builder 3.1.0
• python 3.12
• ansible-navigator 24.2.0 (this specific version to get around ansible-navigator 24.3.2 attempts to load python 3.9 library instead of 3.12 when inspecting EE images #1766

ANSIBLE-NAVIGATOR VERSION

ansible-navigator 24.2.0

CONFIGURATION

---
ansible-navigator:
  execution-environment:
    image: <our custom EE>
    pull:
      policy: missing
  playbook-artifact:
    enable: false
  logging:
    file: ~/ansible-navigator.log

LOG FILE

Ignoring ipaddress: markers 'python_version < "3.3"' don't match your environment
ERROR: Double requirement given: dnspython (from -r /tmp/src/requirements.txt (line 74)) (already in dnspython>=2.0.0 (from -r /tmp/src/requirements.txt (line 63)), name='dnspython')
Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1

STEPS TO REPRODUCE

1. Build Fedora 40 system
2. Run

python3 -m pip install ansible-navigator==24.2.0 --user

3. Define your execution environment as follows:

---
version: 3

build_arg_defaults:
  ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--upgrade'

dependencies:
  galaxy:
    collections:
      - ansible.controller
      - ansible.posix
      - ansible.utils
      - ansible.windows
      - azure.azcollection
      - community.windows
      - microsoft.ad
      - redhat.rhel_system_roles
      - redhat.satellite
      - vmware.vmware_rest
      - community.crypto
      - cloud.vmware_ops
  python:
    - selinux
    - dnspython
    - psutil
    - netaddr
    - pyjwt
  system:
    - findutils [platform:rpm]

images:
  base_image:
    name: registry.redhat.io/ansible-automation-platform-24/ee-minimal-rhel8:latest

additional_build_files:
  - src: /home/bob/.ansible.cfg
    dest: configs

additional_build_steps:
  prepend_galaxy:
    - ADD _build/configs/.ansible.cfg ${HOME}/.ansible.cfg

options:
  package_manager_path: /usr/bin/microdnf

4. Run the following to build the Execution Environment:

 ansible-builder build --verbosity ${EE_VERBOSITY} --prune-images --tag ${EE_NAME}:${EE_VERSION} --tag ${EE_NAME}:latest

EXPECTED RESULTS

It is expected that ansible-builder will be able to resolve the python dependencies for dnspython and build an Execution Environment.

ACTUAL RESULTS

Ignoring ipaddress: markers 'python_version < "3.3"' don't match your environment
ERROR: Double requirement given: dnspython (from -r /tmp/src/requirements.txt (line 74)) (already in dnspython>=2.0.0 (from -r /tmp/src/requirements.txt (line 63)), name='dnspython')
Error: building at STEP "RUN /output/scripts/assemble": while running runtime: exit status 1

ADDITIONAL INFORMATION

When using ansible-builder 3.0.1, the Execution Environment is built successfully without error. ansible-builder 3.1.0 seems to be the problem.

I scanned through the requirements.txt files for all the collections and only found 1 reference to dnspython:

microsoft.ad v1.7.1 requirements.txt

dnspython >= 2.0.0

There should not be 2 entries for dnspython in conflict as I am specifying dnspython in the execution_environment.yml without a version. Further, even if I remove the reference to dnspython from the execution_environment.yml, I still get the error.

[tarmael]

I had the same issue, I found my resolution over here:

ansible/ansible-builder#706

By adding:

dependencies:
  python_interpreter:
    package_system: python3.12
    python_path: /usr/bin/python3.12
  ansible_core:
    package_pip: ansible-core>=2.16,<2.17
  ansible_runner:
    package_pip: ansible-runner

Although I added
package_system: python3.12-devel
instead because it failed to build some pip libraries without Python Development libraries

Ubuntu EE builders may want to use
package_system: python3.12-dev
instead