Skip to content

Commit

Permalink
Merge pull request #1 from TimotheeMathieu/poetry
Browse files Browse the repository at this point in the history
Add support for poetry
  • Loading branch information
TimotheeMathieu authored Dec 14, 2024
2 parents c118374 + aedaade commit bc78a2f
Show file tree
Hide file tree
Showing 12 changed files with 294 additions and 127 deletions.
31 changes: 31 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: tests
on:
pull_request_target:
branches:
- main
types: [closed]
push:
branches:
- main

permissions:
contents: write

jobs:
docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install dependencies
run: |
pip install .
pip install pytest pytest-cov
- name: test
run: |
pytest --cov=guix_env tests
33 changes: 18 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
# Guix & venv environments for reproducible python development

This package give a cli tool to construct and enter environments constructed through guix (for the system-level packages) and pip/venv (for python packages).

Remark that due to the use of pip, the resulting environment is not perfectly reproducible as it would be better to use guix all the way but this way is much simpler when using dev python packages that are not yet packaged in guix.

This package give a cli tool to construct and enter environments constructed through guix (for the system-level packages) and optionally a poetry environment (for python packages).

## Usage
Guix must be installed on the system, see [the guix manual](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html) to do this.
Expand All @@ -13,26 +10,32 @@ pip install git+https://github.com/TimotheeMathieu/guix-env
guix-env create my_env_name
```

This should create a directory in `~/.guix-env` containing files needed for the environment to run. Note that every file generated by guix-env will be saved into `~/.guix-env`. Then to spawn a shell in the environment, do
This should create a directory in `~/.guix-env` containing files needed for the environment to run. Note that every file generated by guix-env will be saved into `~/.guix-env` and removing the environment is as simple as removing this directory (also doable through the `guix-env rm` command). Then to spawn a shell in the environment, do

```
guix-env shell my_env_name
```

The first run may be a bit slow because of guix downloading a bunch of packages but the second run should be faster as guix cache the packages it uses in `/gnu/store` (remark: don't forget to use `guix gc` to clear the store periodically).

Then, you are good to go and do anything you wish in your environment. You are in a python virtual environment and you can install new python packages with pip. To add new guix package, use `guix-env add my_env_name my_package_name` from outside the environment.
Then, you are good to go and do anything you wish in your environment. You are in a python virtual environment that is managed with poetry for reproducibility purpose, to install new python package from inside the environment, use `gep add package_name`, `gep` stands for guix-env-poetry and is just an alias of poetry that install at the right place. To add new guix package, use `guix-env add-guix my_env_name my_package_name` from outside the environment.

One of the qualities of guix-env is its *reproducibility*, you can use the three files `manifest.scm`, `channels.scm` and `requirements.txt` that are in `~/.guix-env/my_env_name` to reproduce the environment using the following command:
```
guix-env create my_env_name --channel-file channels.scm --manifest-file manifest.scm --requirements-file requirements.txt
```
Remark that it is not perfect reproductibility because the requirements.txt file is created using pip whether it would be better to use a lock file generated by `pip-tools` or `poetry`, or even better to use guix as python package manager. For now this is not implemented.


Remark that I made a few opinionated design choices:
- I included some guix and python packages that are convenient for basic shell commands and basic graphical display.
- I use zsh shell in the guix-env environments.
- Every environment has its own .local and .zshrc. They can be accessed in $HOME/.guix_env/env_name
- I do not share the home directory, by default the only directory shared are the current directory and its children (default from guix shell).
- I use the Filesystem Hierarchy Standard (FHS) emulator of guix shell to populate /bin and /lib apropriately for some python compatibility.
- I use a poetry cache specific to guix-env environments. It is shared among the guix envs but not with the host system.

## TODO

- Better documentation
- better requirements.txt -- switch to poetry or pip-tools to handle locks?
- Tests
- Better documentation -- include explanations of how it works: poetry, what do we share (what we do not share, e.g. .local), how to tinker with it, what changes are made...
- Comment more
- Make tests
- Have an alias that install guix_env in a guix shell environment so that we can install & use guix-env in a reproducible maneer and without needing to install anything.
- Feature: rollback, at first this could be through git repo that auto-commit.
- Handle GPU ?
- Feature: rollback, similar to what can be done with guix-home.
- Feature: use tmux inside the env and share tmp to make a sort of daemon. https://stackoverflow.com/questions/16398850/create-new-tmux-session-from-inside-a-tmux-session
Empty file added guix_env/__init__.py
Empty file.
Loading

0 comments on commit bc78a2f

Please sign in to comment.