Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of v2 hashes #18522

Merged
merged 17 commits into from
Nov 12, 2024
Merged

Conversation

bernt-matthias
Copy link
Contributor

@bernt-matthias bernt-matthias commented Jul 10, 2024

Added tests first for the mulled container resolvers here 5b3c4bd

Then fixed (and more tests) here: e7bca83

See details in e7bca83

I'm tempted to backport this to 24.1 in order to make this effective in the multi-package-container repo.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

Copy link
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this can't be right, why did you remove the build parsing ?

@bernt-matthias
Copy link
Contributor Author

I only remove it from the function that parses targets from a string (comma separated list of tools and versions) which is used in build-hash and the other two tools.

The remaining code (e.g. the mulled container resolver etc) keeps the build as part of the version. I think that the tests which I added in my first commit show this (the version part of the hash differs when builds are added, i.e. the build is considered as part of the version when computing the hash).

Wondering if this got a bit confusing when we merged Target and CondaTarget d65721c (guess for conda we parse the build).

@bernt-matthias bernt-matthias force-pushed the topic/v2-hash-fixes branch 3 times, most recently from 1c0354c to 727850b Compare July 22, 2024 10:04
Copy link
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I think this looks good to me now!

@bernt-matthias
Copy link
Contributor Author

Anything else to do here?

to verify if the build is considered as part of the version or not
the build information that may be contained in requirement versions is considered
as part of the version in some places of the code

- mulled container resolvers
- mulled-build-tool
- [planemo](https://github.com/galaxyproject/planemo/blob/b3e12a334e3d358ce30bf2c8c3f8d5a7a7e08136/planemo/conda.py#L116)

in others it isn't:

- mulled-hash
- mulled-build-files
- mulled_build

which are the files that use the `target_str_to_targets` function.

With this change the build info in version strings is always considered
as part of the version.

`mulled-build-files` is used in the multi-package-containers repo which
led to wrong hashes in up to approx 50 tools (check with `grep "=[^,]*=" combinations/* | wc -l`).
an empty set of requirements gives an empty requirement string which
was wrongly parsed to an CondaTarget with empty package name

This triggered a `conda search ''` which took so much memory that other
processes crashed.

The fix is to parse an empty list of CondaTargets for an empty target
string. In addition we check for empty package name during the creation
of the CondaTarget (to make this more easily detectable in the future)
@mvdbeek
Copy link
Member

mvdbeek commented Nov 12, 2024

If github let's me I will merge this 😆

@mvdbeek mvdbeek merged commit 598c3b8 into galaxyproject:dev Nov 12, 2024
54 checks passed
Copy link

This PR was merged without a "kind/" label, please correct.

@bernt-matthias bernt-matthias deleted the topic/v2-hash-fixes branch November 12, 2024 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants