Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce time on tests that run via github actions #135

Open
fgypas opened this issue Dec 15, 2020 · 3 comments
Open

Reduce time on tests that run via github actions #135

fgypas opened this issue Dec 15, 2020 · 3 comments
Labels
enhancement New feature or request

Comments

@fgypas
Copy link
Member

fgypas commented Dec 15, 2020

Is your feature request related to a problem? Please describe.
One of the issues is that tests take quite some time because it requires to run the following workflow https://github.com/panoptes-organization/snakemake_example_workflow that depends on conda.

Describe the solution you'd like
One solution would be to install all dependencies via pip and abandon completely conda

Describe alternatives you've considered
The alternative solution (if the first is not possible) is to use mamba that is very fast

@fgypas fgypas added the enhancement New feature or request label Dec 15, 2020
@vsoch
Copy link
Collaborator

vsoch commented Dec 15, 2020

What about using a container that already has snakemake installed?

@fgypas fgypas changed the title Reduce time on test Reduce time on tests that run via github actions Dec 15, 2020
@fgypas
Copy link
Member Author

fgypas commented Dec 15, 2020

We anyway use docker-compose to deploy panoptes, so I don't see any reason why not use more containers.
https://github.com/panoptes-organization/panoptes/blob/develop/.travis.yml#L20

What takes time is this line: https://github.com/panoptes-organization/panoptes/blob/develop/.travis.yml#L51. What is happening is that for each snakemake rule (https://github.com/panoptes-organization/snakemake_example_workflow/blob/master/Snakefile#L12-L13) we build the dependencies based on conda. Not the main installation of snakemake is the time-consuming step. We could use singularity containers for these steps and run snakemake with --use-singularity.

@vsoch
Copy link
Collaborator

vsoch commented Dec 15, 2020

In the future I hope that snakemake-wrappers can support containers - I did the start of work to add this snakemake/snakemake-wrappers#153 but I don't think it's a priority for Johannes, which is totally understandable!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants