Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: update-pinned-requirements is a little too fast #782

Open
woodruffw opened this issue Sep 28, 2023 · 2 comments
Open

CI: update-pinned-requirements is a little too fast #782

woodruffw opened this issue Sep 28, 2023 · 2 comments
Assignees
Labels
bug Something isn't working component:cicd CI/CD

Comments

@woodruffw
Copy link
Member

Defect during 2.0 release: https://github.com/sigstore/sigstore-python/actions/runs/6342972187/job/17229957076

Run cd install
  ERROR: Could not find a version that satisfies the requirement sigstore==2.0.0 (from versions: 0.0.1rc1, 0.0.1rc2, 0.0.1rc3, 0.1.0, 0.2.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.5.1rc1, 0.5.1rc2, 0.5.1, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.6.7, 0.6.8, 0.7.0, 0.8.3, 0.9.0, 0.10.0, 1.0.0rc1, 1.0.0, 1.1.0, 1.1.1rc1, 1.1.1, 1.1.2rc1, 1.1.2, 2.0.0rc1, 2.0.0rc2, 2.0.0rc3)
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 174, in _add_to_criteria
    raise RequirementsConflicted(criterion)
pip._vendor.resolvelib.resolvers.RequirementsConflicted: Requirements conflict: SpecifierRequirement('sigstore==2.0.0')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 399, in resolve
    raise ResolutionImpossible(e.criterion.information)
pip._vendor.resolvelib.resolvers.ResolutionImpossible: [RequirementInformation(requirement=SpecifierRequirement('sigstore==2.0.0'), parent=None)]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.16/x64/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/click/decorators.py", line [33](https://github.com/sigstore/sigstore-python/actions/runs/6342972187/job/17229957076#step:9:34), in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/piptools/scripts/compile.py", line 659, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/piptools/resolver.py", line 604, in resolve
    is_resolved = self._do_resolve(
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/piptools/resolver.py", line 6[36](https://github.com/sigstore/sigstore-python/actions/runs/6342972187/job/17229957076#step:9:37), in _do_resolve
    resolver.resolve(
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 101, in resolve
    raise error from e
pip._internal.exceptions.DistributionNotFound: No matching distribution found for sigstore==2.0.0
Error: Process completed with exit code 1.

TL;DR is that we run this workflow right after successful PyPI publish, which means that PyPI may not actually be serving the new release just yet.

Possible resolutions:

  • Make this workflow manual-only, include a note in our release process reminding someone to run it
  • Make the workflow poll PyPI for the new version, and only continue once present
  • Make the workflow wait for 1-2 minutes before attempting to do anything (the lazy way)
@woodruffw woodruffw added bug Something isn't working component:cicd CI/CD labels Sep 28, 2023
@di
Copy link
Member

di commented Nov 21, 2024

This is resulting in all our release workflows looking like they've failed: https://github.com/sigstore/sigstore-python/actions/workflows/release.yml

Another thing we could do is set up an RSS-> Issue -> workflow -> PR feed like in https://github.com/di/pip-api/blob/master/.github/workflows/pip-feed.yml

@woodruffw
Copy link
Member Author

Yeah, RSS is probably the right way to go about this -- I think we can remove the current mess here entirely and go with something much simpler like your pip-feed.yml workflow 🙂

@woodruffw woodruffw self-assigned this Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component:cicd CI/CD
Projects
None yet
Development

No branches or pull requests

2 participants