Refactor the code by introducing the concept of an Environment

[ruben-arts]

We want to refactor the code to include the notion of an environment to prepare for multiple environments. An environments provides dependencies (per platform?), the virtual packages (e.g. system requirements) and activation scripts. We need to refactor the code (where appropriate) to use an Environment instead of a Project.

• Check if the lock-file is up to date
• Install an environment

The first step is to have a Project::default() -> Environment function that returns the "default/main" environment and use that throughout the rest of the code.

Install the environment to .pixi/envs/<name> instead of the current .pixi/env folder.

[rahuldave]

What is the use case for this? I'm accustomed to thinking of pixi in the per-folder environment sense: is this for build matrices?

[ruben-arts]

@rahuldave, you should checkout the linked project. We're building a multi env feature. This is for projects that like to specify and lock multiple versions of environments in one project.
If the project doesn't give you enough info for now. Checkout this PR that presents the proposal.

[rahuldave]

Thank you! The PR explains it really well. Essentially a version x platform x special cases ( cuda/cpu/gpu ) matrix which may or may not map to a gha matrix. This is true for builds too, not just testing, so for example if I am looking to use hatchling with pixi ( or a conda builder). So I suppose a builder can be activated in a post install script and be used. But of-course this requires a script to translate to a pyproject.toml to be activated on a per-environment basis or more generally a per environment task ability.

[ruben-arts]

Feel free to add your example use case to that pr! Assuming you mean to say that you want to create a build-hatchling environment this would indeed be possible with that proposal.

[rahuldave]

Yes. Just like we use gha matrices to test we can use them to build right? And one way to do that is to explicitly create a CI build environment..what you said..for each combo. ( and a CD deploy for production systems but those are not libraries and I digress, although pixi in production systems is a great idea ). But my thought is we'll need to build out the task specs a bit ( for which current issues exist) so that tasks can be parametric over environments and don't exponentially increase...

[ruben-arts]

Yeah that will be a next step! Keep following the projects and you will be up to date!

[nmstoker]

Hello - I'm new to pixi (as of about an hour ago) - loving it so far! 🙂

If there are to be environments, might there be scope for some basic metadata, such as the date created / brief description?

Coming from conda, one thing that would occasionally come up after a while on a machine was finding myself with a mess of similar environments where I didn't recall exactly which one was for which particular use. Having the local creation date added automatically and the chance to store a quick summary would be handy. It's obviously possible to write this stuff up somewhere else, but it tends to get separated / lost, so maybe a way to put it in the environment settings might work.

What do you think @ruben-arts ?

[ruben-arts]

Hey @nmstoker,

Good to hear that you like it!

All these environments will be built from the pixi.toml which allows you to create comments to document this information. Also the environment is created at the moment of making the lockfile which means it cannot change without the lockfile changing.

If you want to learn more about the proposed design checkout this document:https://prefix-dev.github.io/pixi/design_proposals/multi_environment_proposal/

I hope this explains why we wouldn't need the metadata. Please let me know if I'm misunderstanding your request!

[nmstoker]

Yes, sounds like comments should work as a way to track that stuff.

had been thinking it would be handy to have info like that show up along with the listing of environments but I guess manual comments are freer and not that much effort to go check by hand (ie opening the pixi.toml file you mention)

Thanks for getting back. I look forward to seeing how things progress.