Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Package compatibility (IDFGH-12037) #13102

Closed
3 tasks done
jiapei100 opened this issue Feb 3, 2024 · 6 comments
Closed
3 tasks done

Package compatibility (IDFGH-12037) #13102

jiapei100 opened this issue Feb 3, 2024 · 6 comments
Labels
Resolution: Won't Do This will not be worked on Status: Done Issue is done internally Type: Bug bugs in IDF

Comments

@jiapei100
Copy link

Answers checklist.

  • I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
  • I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
  • I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

5.1.2

Operating System used.

Linux

How did you build your project?

Command line with Make

If you are using Windows, please specify command line type.

None

What is the expected behavior?

Dependent on

  • requests_file-2.0.0 but NOT requests_file-1.5.1
  • urllib-2.2.0 but NOT urllib3-1.26.18

What is the actual behavior?

ALWAYS install requests_file-1.5.1 and urllib3-1.26.18 by default.

Steps to reproduce.

Just hit ./install.sh

Build or installation Logs.

➜  esp-idf git:(master) ✗ ./install.sh
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Installing ESP-IDF tools
Updating ~/.espressif/idf-env.json
Selected targets are: esp32s3, esp32c3, esp32s2, esp32c2, esp32c6, esp32h2, esp32p4, esp32c5, esp32
Current system platform: linux-amd64
Installing tools: xtensa-esp-elf-gdb, riscv32-esp-elf-gdb, xtensa-esp-elf, riscv32-esp-elf, esp32ulp-elf, openocd-esp32, esp-rom-elfs
Skipping [email protected]_20231023 (already installed)
Skipping [email protected]_20231023 (already installed)
Skipping [email protected]_20240109 (already installed)
Skipping [email protected]_20240109 (already installed)
Skipping [email protected]_20240113 (already installed)
Skipping [email protected] (already installed)
Skipping esp-rom-elfs@20230320 (already installed)
Installing Python environment and packages
Python 3.10.12
pip 24.0 from ~/.local/lib/python3.10/site-packages/pip (python 3.10)
Upgrading pip and setuptools...
Requirement already satisfied: pip in ~/.local/lib/python3.10/site-packages (24.0)
Requirement already satisfied: setuptools in ~/.local/lib/python3.10/site-packages (69.0.3)
Skipping the download of ~/.espressif/espidf.constraints.v5.3.txt because it was downloaded recently.
Installing Python packages
 Constraint file: ~/.espressif/espidf.constraints.v5.3.txt
 Requirement files:
  - ....../esp-idf/tools/requirements/requirements.core.txt
Looking in indexes: https://pypi.org/simple, https://dl.espressif.com/pypi
Ignoring importlib_metadata: markers 'python_version < "3.8"' don't match your environment
Requirement already satisfied: setuptools in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 3)) (69.0.3)
Requirement already satisfied: packaging in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 4)) (23.2)
Requirement already satisfied: click in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 7)) (8.1.7)
Requirement already satisfied: pyserial in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 8)) (3.5)
Requirement already satisfied: cryptography in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 9)) (42.0.2)
Requirement already satisfied: pyparsing in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 10)) (3.1.1)
Requirement already satisfied: pyelftools in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 11)) (0.30)
Collecting idf-component-manager (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 12))
  Using cached idf_component_manager-1.4.2-py2.py3-none-any.whl.metadata (11 kB)
Collecting esp-coredump (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 13))
  Using cached https://dl.espressif.com/pypi/esp-coredump/esp_coredump-1.10.0-py3-none-any.whl (44 kB)
Collecting esptool (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 14))
  Using cached https://dl.espressif.com/pypi/esptool/esptool-4.8.dev1-py3-none-any.whl (388 kB)
Collecting esp-idf-kconfig (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 15))
  Using cached https://dl.espressif.com/pypi/esp-idf-kconfig/esp_idf_kconfig-2.1.0-py3-none-any.whl (44 kB)
Collecting esp-idf-monitor (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 16))
  Using cached esp_idf_monitor-1.4.0-py3-none-any.whl.metadata (4.0 kB)
Collecting esp-idf-nvs-partition-gen (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 17))
  Using cached https://dl.espressif.com/pypi/esp-idf-nvs-partition-gen/esp_idf_nvs_partition_gen-0.1.1-py3-none-any.whl (20 kB)
Collecting esp-idf-size (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 18))
  Using cached esp_idf_size-1.1.0-py3-none-any.whl.metadata (1.6 kB)
Collecting esp-idf-panic-decoder (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 19))
  Using cached https://dl.espressif.com/pypi/esp-idf-panic-decoder/esp_idf_panic_decoder-1.0.1-py3-none-any.whl (18 kB)
Requirement already satisfied: pyclang in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 20)) (0.4.2)
Requirement already satisfied: construct in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 21)) (2.10.70)
Requirement already satisfied: freertos_gdb in ~/.local/lib/python3.10/site-packages (from -r ....../esp-idf/tools/requirements/requirements.core.txt (line 24)) (1.0.2)
Requirement already satisfied: cffi>=1.12 in ~/.local/lib/python3.10/site-packages (from cryptography->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 9)) (1.16.0)
Requirement already satisfied: requests<3 in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (2.31.0)
Collecting urllib3<2 (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12))
  Using cached https://dl.espressif.com/pypi/urllib3/urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Collecting requests-file<2 (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12))
  Using cached https://dl.espressif.com/pypi/requests-file/requests_file-1.5.1-py2.py3-none-any.whl (3.7 kB)
Requirement already satisfied: requests-toolbelt in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (1.0.0)
Requirement already satisfied: schema in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (0.7.5)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (1.16.0)
Requirement already satisfied: tqdm<5 in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (4.66.1)
Requirement already satisfied: colorama in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (0.4.6)
Requirement already satisfied: pyyaml>5.2 in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (6.0.1)
Requirement already satisfied: cachecontrol>0.12.6 in ~/.local/lib/python3.10/site-packages (from cachecontrol[filecache]>0.12.6; python_version >= "3.6"->idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (0.14.0)
Requirement already satisfied: contextlib2>0.6.0 in ~/.local/lib/python3.10/site-packages (from idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (21.6.0)
Requirement already satisfied: pygdbmi>=0.9.0.2 in ~/.local/lib/python3.10/site-packages (from esp-coredump->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 13)) (0.11.0.0)
Requirement already satisfied: bitstring>=3.1.6 in ~/.local/lib/python3.10/site-packages (from esptool->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 14)) (4.1.4)
Requirement already satisfied: ecdsa>=0.16.0 in ~/.local/lib/python3.10/site-packages (from esptool->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 14)) (0.18.0)
Requirement already satisfied: reedsolo<1.8,>=1.5.3 in ~/.local/lib/python3.10/site-packages (from esptool->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 14)) (1.7.0)
Requirement already satisfied: intelhex in ~/.local/lib/python3.10/site-packages (from esptool->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 14)) (2.3.0)
Requirement already satisfied: kconfiglib>=13.7.1 in ~/.local/lib/python3.10/site-packages (from esp-idf-kconfig->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 15)) (14.1.0)
Requirement already satisfied: rich in ~/.local/lib/python3.10/site-packages (from esp-idf-size->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 18)) (13.7.0)
Requirement already satisfied: bitarray<3.0.0,>=2.8.0 in ~/.local/lib/python3.10/site-packages (from bitstring>=3.1.6->esptool->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 14)) (2.9.2)
Requirement already satisfied: msgpack<2.0.0,>=0.5.2 in ~/.local/lib/python3.10/site-packages (from cachecontrol>0.12.6->cachecontrol[filecache]>0.12.6; python_version >= "3.6"->idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (1.0.7)
Requirement already satisfied: filelock>=3.8.0 in ~/.local/lib/python3.10/site-packages (from cachecontrol[filecache]>0.12.6; python_version >= "3.6"->idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (3.13.1)
Requirement already satisfied: pycparser in ~/.local/lib/python3.10/site-packages (from cffi>=1.12->cryptography->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 9)) (2.21)
Requirement already satisfied: charset-normalizer<4,>=2 in ~/.local/lib/python3.10/site-packages (from requests<3->idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in ~/.local/lib/python3.10/site-packages (from requests<3->idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (3.6)
Requirement already satisfied: certifi>=2017.4.17 in ~/.local/lib/python3.10/site-packages (from requests<3->idf-component-manager->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 12)) (2024.2.2)
Requirement already satisfied: markdown-it-py>=2.2.0 in ~/.local/lib/python3.10/site-packages (from rich->esp-idf-size->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 18)) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in ~/.local/lib/python3.10/site-packages (from rich->esp-idf-size->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 18)) (2.17.2)
Requirement already satisfied: mdurl~=0.1 in ~/.local/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->esp-idf-size->-r ....../esp-idf/tools/requirements/requirements.core.txt (line 18)) (0.1.2)
Using cached idf_component_manager-1.4.2-py2.py3-none-any.whl (140 kB)
Using cached esp_idf_monitor-1.4.0-py3-none-any.whl (44 kB)
Using cached esp_idf_size-1.1.0-py3-none-any.whl (44 kB)
Installing collected packages: urllib3, esp-idf-panic-decoder, esp-idf-kconfig, requests-file, esptool, esp-idf-size, esp-idf-nvs-partition-gen, esp-coredump, idf-component-manager, esp-idf-monitor
  Attempting uninstall: urllib3
    Found existing installation: urllib3 2.2.0
    Not uninstalling urllib3 at ~/.local/lib/python3.10/site-packages, outside environment ~/.espressif/python_env/idf5.3_py3.10_env
    Can't uninstall 'urllib3'. No files were found to uninstall.
  Attempting uninstall: requests-file
    Found existing installation: requests-file 2.0.0
    Not uninstalling requests-file at ~/.local/lib/python3.10/site-packages, outside environment ~/.espressif/python_env/idf5.3_py3.10_env
    Can't uninstall 'requests-file'. No files were found to uninstall.
Successfully installed esp-coredump-1.10.0 esp-idf-kconfig-2.1.0 esp-idf-monitor-1.4.0 esp-idf-nvs-partition-gen-0.1.1 esp-idf-panic-decoder-1.0.1 esp-idf-size-1.1.0 esptool-4.8.dev1 idf-component-manager-1.4.2 requests-file-1.5.1 urllib3-1.26.18
All done! You can now run:

  . ./export.sh

➜  esp-idf git:(master) ✗

More Information.

  • OS: Ubuntu 22.04.3
  • Python: Python 3.10.12
@jiapei100 jiapei100 added the Type: Bug bugs in IDF label Feb 3, 2024
@espressif-bot espressif-bot added the Status: Opened Issue is new label Feb 3, 2024
@github-actions github-actions bot changed the title Package compatibility Package compatibility (IDFGH-12037) Feb 3, 2024
@jiapei100
Copy link
Author

jiapei100 commented Feb 4, 2024 •
edited
Loading

It looks idf.py is unable to find the installed urllib3-1.26.18 and requests_file-1.5.1 under folder ~/.espressif/python_env/idf5.3_py3.10_env/lib/python3.10/site-packages, but under ~/.local/lib/python3.10/site-packages.

➜  esp-idf git:(master) ✗ idf.py --version
The following Python requirements are not satisfied:
Requirement 'urllib3<2' was not met. Installed version: 2.2.0
Requirement 'requests-file<2' was not met. Installed version: 2.0.0
To install the missing packages, please run "install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: ~/.espressif/python_env/idf5.3_py3.10_env
    Python interpreter used: ~/.espressif/python_env/idf5.3_py3.10_env/bin/python
Constraint file: ~/.espressif/espidf.constraints.v5.3.txt
Requirement files:
 - ....../esp-idf/tools/requirements/requirements.core.txt
Python being checked: ~/.espressif/python_env/idf5.3_py3.10_env/bin/python

ESP-IDF v5.3-dev-1724-ga5b261f699-dirty
➜  site-packages pwd
~/.espressif/python_env/idf5.3_py3.10_env/lib/python3.10/site-packages
➜  site-packages ls
esp_coredump                     esp_idf_monitor-1.4.0.dist-info            esp_idf_size-1.1.0.dist-info           idf_component_tools            requests_file.py
esp_coredump-1.10.0.dist-info    esp_idf_nvs_partition_gen                  espsecure                              kconfcheck                     test
espefuse                         esp_idf_nvs_partition_gen-0.1.1.dist-info  esptool                                kconfgen                       tests
esp_idf_kconfig                  esp_idf_panic_decoder                      esptool-4.8.dev1.dist-info             kconfserver                    urllib3
esp_idf_kconfig-2.1.0.dist-info  esp_idf_panic_decoder-1.0.1.dist-info      idf_component_manager                  __pycache__                    urllib3-1.26.18.dist-info
esp_idf_monitor                  esp_idf_size                               idf_component_manager-1.4.2.dist-info  requests_file-1.5.1.dist-info
➜  site-packages 
➜  ~ pip show urllib3
Name: urllib3
Version: 2.2.0
Summary: HTTP library with thread-safe connection pooling, file post, and more.
Home-page: 
Author: 
Author-email: Andrey Petrov <[email protected]>
License: 
Location: ~/.local/lib/python3.10/site-packages
Requires: 
Required-by: requests
➜  ~ pip show requests_file
Name: requests-file
Version: 2.0.0
Summary: File transport adapter for Requests
Home-page: 
Author: 
Author-email: David Shea <[email protected]>
License: Apache 2.0
Location: ~/.local/lib/python3.10/site-packages
Requires: requests
Required-by: 
➜  ~

@jiapei100
Copy link
Author

It looks the issue is from idf-component-manager, should be just in the setup.py:
espressif/idf-component-manager#50

@jiapei100
Copy link
Author

Sorry that I have to re-open this issue, because I now met the same issue as #10823, and my error message is:

[2/2] idf (5.3.0)
-- Project sdkconfig file ....../esp-who/examples/human_face_detection/lcd/sdkconfig
~/.espressif/python_env/idf5.3_py3.10_env/bin/python: No module named kconfgen.__main__; 'kconfgen' is a package and cannot be directly executed
CMake Error at /opt/Espressif/esp-idf/tools/cmake/kconfig.cmake:209 (message):
  Failed to run kconfgen
  (~/.espressif/python_env/idf5.3_py3.10_env/bin/python;-m;kconfgen;--list-separator=semicolon;--kconfig;/opt/Espressif/esp-idf/Kconfig;--sdkconfig-rename;/opt/Espressif/esp-idf/sdkconfig.rename;--config;....../esp-who/examples/human_face_detection/lcd/sdkconfig;--defaults;....../esp-who/examples/human_face_detection/lcd/sdkconfig.defaults;--defaults;....../esp-who/examples/human_face_detection/lcd/sdkconfig.defaults.esp32s3;--env-file;....../esp-who/examples/human_face_detection/lcd/build/config.env).
  Error 1
Call Stack (most recent call first):
  /opt/Espressif/esp-idf/tools/cmake/build.cmake:615 (__kconfig_generate_config)
  /opt/Espressif/esp-idf/tools/cmake/project.cmake:710 (idf_build_process)
  CMakeLists.txt:8 (project)


-- Configuring incomplete, errors occurred!

@jiapei100 jiapei100 reopened this Feb 5, 2024
This was referenced Feb 5, 2024
Closed
Closed
@dobairoland
Copy link
Collaborator

I don't remember any similar issue when packages installed in ~/.local/ would block installing different versions in the virtual environment. Did you create ~/.espressif/python_env/idf5.3_py3.10_env on your own? Could you please remove it and re-create it by re-running install.sh?

This is just an irrelevant side note. Your ESP-IDF version is not 5.1.2 but the the upcoming v5.3 (master branch).

@dobairoland
Copy link
Collaborator

Have you set any of the ESP-IDF environment variables permanently in your system (e.g. IDF_PYTHON_ENV_PATH)? The output of install.sh doesn't seem right to me. It looks as if the Python environment would be redirected to ~/.local/lib/python3.10. After that, the export script switches to the ESP-IDF environment (based on your idf.py --version) correctly. That is why you getting these issues. The install script updates a different environment.

@jiapei100
Copy link
Author

My bad... I got 2 versions of kconfgen installed, one under ~/.local, the other under ~/.espressif .

Thank you

@espressif-bot espressif-bot added Status: Done Issue is done internally Resolution: Won't Do This will not be worked on and removed Status: Opened Issue is new labels Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Resolution: Won't Do This will not be worked on Status: Done Issue is done internally Type: Bug bugs in IDF
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jiapei100 @dobairoland @espressif-bot