Skip to content

Commit

Permalink
Update README for 2.0 version
Browse files Browse the repository at this point in the history
  • Loading branch information
poquirion committed Jun 7, 2021
1 parent b067b43 commit d2f055e
Showing 1 changed file with 13 additions and 24 deletions.
37 changes: 13 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@

You can use Genpipes in a Container (GiaC) to run Genpipes on a single machine, on a a Torque/PBS cluster or on a SLURM cluster.

If FUSE and [Singularity](https://singularity.lbl.gov/index.html) is installed on your machine you are all set, a simple user with no special privilege is enough.
If [Singularity](https://github.com/hpcng/singularity) is installed on your LINUX machine you are all set, a simple user with no special privilege is enough.

While you can use (GiaC) to debug and develop Genpipes on you machine on you machine, [Genpipes](https://bitbucket.org/mugqic/genpipes/src/master/README.md) is design to run analysis on Super Computers.
While you can use (GiaC) to debug and develop Genpipes on you machine on your machine laptop, [Genpipes](https://bitbucket.org/mugqic/genpipes/src/master/README.md) is design to run analysis on Super Computers.


## Install a compatible container technology on your machine
Follow installation procedure from te [Singularity install page](https://sylabs.io/guides/3.5/admin-guide/installation.html)
Follow installation procedure from te [Singularity install page](ttps://github.com/hpcng/singularity/blob/master/INSTALL.md)


You can also [Read the genpipes documentation](https://bitbucket.org/mugqic/genpipes)
You can also [Read the genpipes documentation](https://genpipes.readthedocs.io/)

## What exactly is avalable in that container?

The full tested and integrated C3G/MUGQIC software stack.
The full tested and integrated C3G/MUGQIC software stack, a complete set of genomics references.
[For more details: http://www.computationalgenomics.ca/cvmfs-modules/](http://www.computationalgenomics.ca/cvmfs-modules/)


Expand All @@ -37,39 +37,28 @@ You can now configuere the `$WORKDIR/genpipes/resources/container/etc/wrapper.co
```
# GEN_SHARED_CVMFS should have a sufficient amount of space to load full reference files
export GEN_SHARED_CVMFS=$HOME/cvmfs
TMPDIR=/tmp
export UMOUNT_RETRY=5
BIND_LIST=
```

`GEN_SHARED_CVMFS` will hold a cache for GiaC [CVMFS](https://cernvm.cern.ch/portal/filesystem) system, it will hold the genomes and software being used by Genpipes. This folder will grow with Genpipes usage. You can delete in in between usage, but keep in mind that once deleted it will need to be rebuild by downloading data form the internet.
`GEN_SHARED_CVMFS` will hold a cache for GiaC [CVMFS](https://cernvm.cern.ch/portal/filesystem) system, it will hold the genomes and software being used by Genpipes. This folder will grow with Genpipes usage. You can delete it in between usage, but keep in mind that once deleted it will need to be rebuild by downloading data form the internet.

`BIND_LIST` is a list of file system, separated by comma, you need Genpipes to have access to, by default, only your $HOME is mounted. For example if you are on an HPC system with a `/scratch` and `/data` space, you would have `BIND_LIST=/scratch,/data`. The string will be fed to Singularity `--bind` option, see `singularity --help` for more details.

`TMPDIR` place to store GiaC temp files. These are a mainly metadata of what will be stored in the `GEN_SHARED_CVMFS` folder. The volume of data in this folder will be small.

`UMOUNT_RETRY` GiaC will mount a [CVMFS](https://cernvm.cern.ch/portal/filesystem) in the `TMPDIR` folder, once GiaC is done, the folder is unmounted up to `UMOUNT_RETRY` time, usually two try does the trick, but 5 is safer.


You do not need any other setup on your machine.

## PIPELINE USAGE

# On SLURM HPC

Created an ini file that fits your system and for the pipeline(s) you want to use.


add the `--wrap` option when running the pipelines

# On PBS/torque flavored HPC
The GenPipes documentation page is here:
https://genpipes.readthedocs.io/

Created an ini file that fits your system and for the pipeline(s) you want to use.
# On SLURM or PBS/torque HPC

add the `--wrap` option when running the pipelines
Create an ini file that fits your system and for the pipeline(s) you want to use.

Add the `-j {pbs,slurm}` option to fit your scheduler then the `--wrap` options so GenPipes with wrap all its command with the container instrumentation.

# On a single machine.

Run the pipeline with the `--wrap`, `-j batch` and `--no-json` options!

Just run the pipeline with the `--wrap` and `-j batch` and `--no-json` options!
You can also run the `./genpipes/resources/container/bin/container_wrapper.sh` command to get inside the container with the right configuration. You will then have access to all the GenPipes tools be able to run them directly inside the container, on a single host without the `--wrap` option.

0 comments on commit d2f055e

Please sign in to comment.