Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[24.1] Remove defaults channel for conda usage #18859

Merged
merged 7 commits into from
Oct 22, 2024
Merged

[24.1] Remove defaults channel for conda usage #18859

merged 7 commits into from
Oct 22, 2024

Conversation

bernt-matthias
Copy link
Contributor

To get rid of the potential problem with the default channel in planemo usage and up-to-date installations I would suggest a backport. Instead of 24.1 we could also got further back.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@github-actions github-actions bot added this to the 24.1 milestone Sep 20, 2024
@nsoranzo
Copy link
Member

Not sure if this can be considered a bugfix and therefore qualify for backporting, it's a bit of a gray area.
I am not sure changing this default is a good idea without first announcing it on the release notes.

@jmchilton
Copy link
Member

This seems like a troubling thing to backport to me, I would be pretty upset if I was an admin and this just changed in a stable bug fix. Maybe I don't understand though 😢. If you want to encourage people running older Galaxies to do this - isn't encouraging them to set the config values that way a more appropriate, intentional change?

@nsoranzo
Copy link
Member

If you want to encourage people running older Galaxies to do this - isn't encouraging them to set the config values that way a more appropriate, intentional change?

Agreed, we could write a short message to post to the relevant admin Matrix channels and mailing lists.

@bernt-matthias
Copy link
Contributor Author

I would be pretty upset if I was an admin

Yeah .. I primarily thought about planemo. If I remember right planemo uses the master branch by default. Would it be an option to deploy these commits to master?

For admins a note to

  1. change channels
  2. switch to miniforge
  3. and (maybe) to re-create all conda envs that made use of the defaults channel (.. ugh) might indeed be better.

Maybe I don't understand though

Sorry. Here is the story (as far as I got it): The company behind conda (silently) changed their terms of service in 2020. Basically the default channel is not free anymore for institutions with more than 200 people and there have been reports that they are pushing this policy. They charge 50$ per person and year (also for those persons in your institution that don't use conda) and also back until 2020 .. troublesome for an institution like mine, hospitals and also universities.

My conclusion for planemo is that it's not safe to use as long as we use miniconda and the defaults channel.

For point 3 of my list above I'm not entirely sure if this is needed. Maybe not for environments older than 2020. Our HPC admins, e.g., will remove all conda envs that made use of the defaults channel (probably to be on the safe side). And I'm really wondering about the containers at biocontainers...

@jmchilton
Copy link
Member

If this about Planemo - I feel like the appropriate change is to override the default when launching Galaxy from Planemo. It already has the ability to set these options right? Maybe instead of leaving them null or whatever we do to defer to Galaxy we set them to explicit values for a few releases?

@bgruening
Copy link
Member

I would also propose putting it into dev. If its important for planemo we can for the time being point to a backported branch (this branch here), let planemo depend on a pre-release, or just change the defaults as John mentioned.

With the strict-channel-order we should not consume anything from the "default" channel, so we can wait IMHO for the next release.

@nsoranzo
Copy link
Member

I would also propose putting it into dev.

To clarify, it's already in dev, this is a backport.

@bernt-matthias
Copy link
Contributor Author

bernt-matthias commented Sep 30, 2024 •
edited
Loading

I would also propose putting it into dev.

It's already in dev.

If its important for planemo we can for the time being point to a backported branch (this branch here), let planemo depend on a pre-release, or just change the defaults as John mentioned.

If we use this branch, installations like usegalaxy would use the change. I think is that is what @jmchilton wants to avoid (for good reasons). Creating another brach from 24.1 seems to be the best solultion for me. But we would need a new (pre)release of the packages and a new release of planemo.

With the strict-channel-order we should not consume anything from the "default" channel, so we can wait IMHO for the next release.

Problem is that we still install miniconda which constructs its base environment using the defaults channel.

This worries me for instance for the CI in our company maintained tool repo and for my usage in planemo test.

See also recent discussion on matrix.

@martenson
Copy link
Member

wondering about the containers at biocontainers

are they possibly using defaults ?

@nsoranzo
Copy link
Member

If you strip out the change to lib/galaxy/tool_util/deps/resolvers/conda.py, I believe the rest shouldn't affect an existing 24.1 installation (assuming it's not using the sample galaxy.yml.sample file).

@bernt-matthias
Copy link
Contributor Author

wondering about the containers at biocontainers

are they possibly using defaults ?

Yep. Some of them do.

If you strip out the change to lib/galaxy/tool_util/deps/resolvers/conda.py, I believe the rest shouldn't affect an existing 24.1 installation (assuming it's not using the sample galaxy.yml.sample file).

That might be a good compromise. Then planemo would be covered by its --channels argument.

bernt-matthias and others added 7 commits October 21, 2024 21:31
@bernt-matthias
remove defaults channel for conda
4e81a78 
https://www.theregister.com/2024/08/08/anaconda_puts_the_squeeze_on/

According to the FAQs on https://repo.anaconda.com/pkgs/ this includes
`pkgs/main`, `pkgs/r`, and `pkgs/msys2`.

Seems also to be a good idea because for `mulled_channels` we apparently never used it (consistency).
@bernt-matthias
update osx startup workflow
b71b6ef 
to use miniforge
@bernt-matthias
deprecate 32bit conda installation option
14f382e 
can be installed manually by affected users
@nsoranzo @bernt-matthias
Remove conda defaults channel from more places
af9b9ef
@bernt-matthias
add miniforge3 to prefix search
b99e3c8
@bernt-matthias @nsoranzo
also add home for default return
57b3666 
Co-authored-by: Nicola Soranzo <[email protected]>
@bernt-matthias
reset default channels
56fae9b
@bernt-matthias bernt-matthias force-pushed the topic/conda-defaults-24.1 branch from 1ba1d4b to 56fae9b Compare October 21, 2024 19:32
@bernt-matthias
Copy link
Contributor Author

I reverted the changes to lib/galaxy/tool_util/deps/resolvers/conda.py as suggested by @nsoranzo

@nsoranzo nsoranzo merged commit 2cedc9a into galaxyproject:release_24.1 Oct 22, 2024
46 of 49 checks passed
@github-actions GitHub Actions
Copy link

This PR was merged without a "kind/" label, please correct.

@bernt-matthias bernt-matthias deleted the topic/conda-defaults-24.1 branch October 23, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nsoranzo nsoranzo nsoranzo approved these changes

Assignees
No one assigned
Labels
area/admin area/documentation area/tool-dependencies area/tool-framework kind/bug
Projects
None yet
Milestone
24.1
Development

Successfully merging this pull request may close these issues.
5 participants
@bernt-matthias @nsoranzo @jmchilton @bgruening @martenson