Fix pip related requirements issue

[pr4deepr]

the pip bug that was causing installation issues has been fixed as of pip 23.3, so if you are interested we could fix all the requirements in this repo and just ask users to update their pip?

[pr4deepr]

@multimeric
Could you add a bit more detail here and link to the relevant issues/code?

[multimeric]

Yep. The PR that fixed this is pypa/pip#12095. You'll note that one of my issues is linked to that PR. The one-sentence summary they use in the changelog is "Improve extras resolution for multiple constraints on same base package."

[multimeric]

Specifically, the things we can do once we enforce a newer pip are:

• Add napari-aicsimageio back as a dependency
• Remove a lot of the pointless dependency constraints I had to add
• Installation will be quicker

[thawn]

I also ran into installation issues with pip v23.3.1 on Windows
here is what I did:

mamba create -n lattice python=3.10
mamba activate lattice
pip install napari-lattice

The installation gets stuck with the following message:

INFO: pip is looking at multiple versions of scikit-image to determine which version is compatible with other requirements. This could take a while.
Collecting scikit-image!=0.20.0,>=0.18.0 (from pyclesperanto-prototype>=0.20.0->napari-lattice)
  Using cached scikit_image-0.21.0-cp311-cp311-win_amd64.whl.metadata (14 kB)
  Using cached scikit-image-0.19.3.tar.gz (22.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... |

p.s. I am not sure if this is the same bug. let me know and I'll open a new one

[pr4deepr]

Hi @thawn
Good to see you here. I ran installation on my Windows 10 machine using pip v23.3.1 and it seemed to install without any hiccups.
Could you verify your pip version using pip --version ?
If its still giving issues, could you try clearing the cache in conda? https://docs.conda.io/projects/conda/en/latest/commands/clean.html

Cheers
Pradeep

[thawn]

Hi @pr4deepr,
thanks for the quick reply and warm welcome - same back to you ;-)

I can confirm that installation works on a windows server 2019 machine. However, the problem occurs on Windows 11.
The problem was persistent on Windows 11 until I forced pip to stick to scikit-image>=0.20.0:

pip install napari-lattice "scikit-image>=0.20.0"

maybe consider making that part of the dependencies?

Cheers,

Till

Here is what I did in detail

I now cleaned the pip and conda/mamba caches and set up a fresh environment with python 3.11

pip cache purge
mamba clean -ay
mamba create -n lattice python=3.11
mamba activate lattice

this gives me pip 23.3.2:

pip --version
pip 23.3.2 from C:\Users\Till\mambaforge\envs\lattice\Lib\site-packages\pip (python 3.11)

just to be on the safe side, I cleaned the caches again:

pip cache purge
mamba clean -ay

and then installed napari-lattice:

pip install napari-lattice

this again tries to build several versions of scikit_image:

INFO: pip is looking at multiple versions of scikit-image to determine which version is compatible with other requirements. This could take a while.
Collecting scikit-image!=0.20.0,>=0.18.0 (from pyclesperanto-prototype>=0.20.0->napari-lattice)
  Downloading scikit_image-0.21.0-cp311-cp311-win_amd64.whl.metadata (14 kB)
  Downloading scikit-image-0.19.3.tar.gz (22.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.2/22.2 MB 46.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... /

and ultimately fails:

      INFO: No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
      INFO: customize MSVCCompiler
      INFO: customize MSVCCompiler using ConditionalOpenMP
      INFO: CCompilerOpt.cc_test_flags[1086] : testing flags (/O2)
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      INFO: CCompilerOpt.cache_flush[864] : write cache to path -> C:\Users\Till\AppData\Local\Temp\pip-install-uciz7o_h\scikit-image_8d7cc58706bc48e686fa307b46169a35\build\temp.win-amd64-3.11\Release\ccompiler_opt_cache_ext.py
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for scikit-image
  Building wheel for asciitree (setup.py) ... done
  Created wheel for asciitree: filename=asciitree-0.3.3-py3-none-any.whl size=5047 sha256=808f5dd25749e43fcc8e400064efbab51b2e85a09eb33bd17c9f5821f12cb27f
  Stored in directory: c:\users\till\appdata\local\pip\cache\wheels\71\c1\da\23077eb3b87d24d6f3852ed1ed1a1ac2d3c885ad6ebd2b4a07
Successfully built read-roi pims reikna asciitree
Failed to build scikit-image
ERROR: Could not build wheels for scikit-image, which is required to install pyproject.toml-based projects

I checked and Microsoft C++ Bild Tools 2022 is installed (version 17.8.3)
I suspect this is a bug in scikit-image 0.19.3

and io and behold if I do

pip install napari-lattice "scikit-image>=0.20.0"

it works.

[thawn]

p.s. I also found this: https://forum.image.sc/t/pip-install-scikit-image-fails-windows11-python3-11/73311

but I guess restricting to scikit-image>=0.20.0 is better than restricting to python<3.11

[multimeric]

Hi Till. My feeling is that this is a different bug, but also related to installation. I think that your environment is not set up to compile sk-image from source, so only binary (pre-compiled) packages are working for you. 0.19.3 specifically is failing for you because there are no binary wheels for that release, for your version of Python because Python 3.11 didn't exist at the time. I don't use Windows but I would double check that the C++ compiler is configured in such a way that Pip is able to find it.

It might be possible to loosen some dependencies to get this to work with pure binary packages, but I can't guarantee it because it depends on what our dependencies do, which aren't in our control.

[multimeric]

Feedback from @pr4deepr:

Also, during installation,dask-distributed (distributed==2023.5.0) and xarray (xarray<2023.02.0,>=0.16.1) takes a bit of time to resolve the versions.

[multimeric]

Just noting that it takes 10:57 minutes to install the thin-wrapper branch. If I add napari-aicsimageio back in as a dependency, it now succeeds, but takes 30:34 to do so. This is with the latest pip: 24.2.

The particular packages it gets stuck on are:

• imageio
• xarray
• aicsimageio
• dask

I might be able to reduce the installation time by enforcing a smaller range of versions for these packages, but this will reduce compatibility somewhat. This means it will not work as well inside an existing environment that has other packages as well.

[multimeric]

One proposed solution to this was to build a conda package, although as discussed here, this isn't particularly simple.

However something that's very promising is uv, a package manager that is an alternative to pip. I just tested it with uv add ./core ./plugin which is equivalent to the command above where I tested pip. It took under 10 seconds, which is remarkable considering pip takes 10 minutes!

[multimeric]

TODO:

• Add uv to docs
• Talk about napari-aicsimageio to docs
• Ask upstream to fix conda packages