stop using requirementslib models

[matteius]

The requirementslib models are a nested mess. I fixed the issue of wheel extras not being installed and broke a bunch of other stuff in the process the other night, but it dawned on me that there is a lot of translating between pip lines and data models and back and forth and it really is messy and doesn't make much sense.

This is the start of re-establishing what interfaces are actually needed to get between Pipfile, Pipfile.lock and pip lines. The preferred data model to use is pip's InstallRequirement since requirementslib was using it under the hood anwyay as ireq.

There are still a number of edge cases to work out, but I am opening this draft PR now because I want to see if I actually broke the test runner or just my local environment.

At a high level this is what I am thinking but it requires dropping --skip-lock the rest of the way first:

Fixes #5817
Fixes #5789
Fixes #5696
Fixes #5626
Fixes #5595
Fixes #5490
Fixes #5716 (probably)

The checklist

• Associated issue
• A news fragment in the news/ directory to describe this fix with the extension .bugfix.rst, .feature.rst, .behavior.rst, .doc.rst. .vendor.rst. or .trivial.rst (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

[matteius]

Thanks @DanielSwain I think that will be important to sort out as well, to be honest, it was a short cut I took to get things working which makes the two big things I know about that need more work here are:

• Revisit how to lock the hash for all VCS reqs
• local VCS+file:// install requirements (however this doesn't seem to work on the main branch either, but if possible, it would be cool to get it working).
  EDIT: Actually, are local VCS+file:// lines even really possible? From reading the existing code I thought maybe they were, but on pip I don't see it working either:

$ pip install git+file://../../pipenv@main
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\annotated_types-0.4.0-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\cerberus-1.3.4-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\pep517-0.13.0-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\pip-23.1.2-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\plette-0.4.4-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\requirementslib-2.3.1.dev0-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
DEPRECATION: Loading egg at c:\users\matte\appdata\local\programs\python\python311\lib\site-packages\tomlkit-0.11.8-py3.11.egg is deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use pip for package installation..
Collecting git+file://../../pipenv@main
  Cloning file://../../pipenv (to revision main) to c:\users\matte\appdata\local\temp\pip-req-build-dwwna63w
  Running command git clone --filter=blob:none --quiet file://../../pipenv 'C:\Users\matte\AppData\Local\Temp\pip-req-build-dwwna63w'
  fatal: '//../../pipenv' does not appear to be a git repository
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  error: subprocess-exited-with-error

  × git clone --filter=blob:none --quiet file://../../pipenv 'C:\Users\matte\AppData\Local\Temp\pip-req-build-dwwna63w' did not run successfully.
  │ exit code: 128
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet file://../../pipenv 'C:\Users\matte\AppData\Local\Temp\pip-req-build-dwwna63w' did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[DanielSwain]

@matteius When you get hashing on VCS repos re-implemented, just let me know. I'd be glad to test that for you.

[matteius]

@DanielSwain I got the VCS lockfile hashing sorted out now. 👍

[kalebmckale]

Hey @matteius I'm wondering if there is a way to get more details/output when a dependency resolution check fails. I'm having an issue getting Pipfile with particular private PyPI registry packages to lock, and even with --verbose I cannot figure out why this is the case. If I pip install the packages, there are no dependency conflicts. If I create a requirements file using pip freeze after manually installing everything I can then create a lock file from the requirements file. However, the standard unpinned Pipfile will not lock. Any hidden options or ideas would be greatly helpful.

[matteius]

@kalebmckale I just pushed a commit to this branch which gets the -v or --verbose flag working with the right level of pip output from the resolver now. Please give it a try.

[kalebmckale]

@matteius I tested with the version. The verbose output was very helpful. I think I know what the problem is, but I'm not sure why it's happening. Is this the thread to discuss it or should I create a new issue since it seems to be behavior independent of the current changes?

[matteius]

@kalebmckale please open a new issue for that then.

[kalebmckale]

@kalebmckale please open a new issue for that then.

Done!