Skip to content

Commit

Permalink
Merge pull request #275 from NBISweden/js-lectures
Browse files Browse the repository at this point in the history
More lecture updates
  • Loading branch information
fasterius authored Nov 21, 2024
2 parents 43ed0fb + 70c5729 commit 97ab1b9
Show file tree
Hide file tree
Showing 3 changed files with 58 additions and 100 deletions.
70 changes: 28 additions & 42 deletions lectures/conda/conda.qmd
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
---
title: 'Managing software environments with <br> <img src="images/conda-logo.png" style="width:50%;padding: 25px 25px 25px 25px;">'
format: revealjs
format:
revealjs:
mermaid:
theme: neutral
execute:
echo: false
jupyter:
kernelspec:
display_name: Python 3
language: python
name: python3
---

## The problem
Expand All @@ -19,38 +17,33 @@ _Full reproducibility requires the possibility to recreate the system that was
originally used to generate the results._
:::

## Conda is a package, dependency, and environment manager {auto-animate="true"}
## Conda is a package, dependency, and environment manager

- [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.)

::: {data-id="flowchart1"}
```{mermaid}
flowchart LR
multiqc(multiqc)
```
:::

## Conda is a package, dependency, and environment manager {auto-animate="true"}
## Conda is a package, dependency, and environment manager

- [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.)
- [dependency]{.green}: other software required by a package

::: {data-id="flowchart1"}
```{mermaid}
flowchart LR
multiqc(multiqc) -.-> numpy(numpy)
multiqc -.-> matplotlib(matplotlib)
multiqc -.-> python(python)
```
:::

## Conda is a package, dependency, and environment manager {auto-animate="true"}
## Conda is a package, dependency, and environment manager

- [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.)
- [dependency]{.green}: other software required by a package
- dependencies in turn have their own dependencies

::: {data-id="flowchart1"}
```{mermaid}
flowchart LR
multiqc(multiqc) -.-> numpy(numpy)
Expand All @@ -62,16 +55,14 @@ flowchart LR
numpy -.-> python
numpy -.-> libcxx(libcxx)
```
:::

## Conda is a package, dependency, and environment manager {auto-animate="true"}
## Conda is a package, dependency, and environment manager

- [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.)
- [dependency]{.green}: other software required by a package
- dependencies in turn have their own dependencies
- [environment]{.green}: a distinct collection of packages

::: {data-id="flowchart1"}
```{mermaid}
flowchart LR
subgraph environment
Expand All @@ -87,13 +78,11 @@ flowchart LR
numpy -.-> libcxx(libcxx)
end
```
:::

## Conda channels {auto-animate=true}

## Conda channels

[Channels]{.green} are remote directories containing packages

::: {data-id="m1"}
```{mermaid}
flowchart TD
ch1[(channel1)] --- p1[package1]
Expand All @@ -104,16 +93,15 @@ flowchart TD
ch2[(channel2)] --- p5[package5]
ch2[(channel2)] --- p6[package6]
```
:::

## Conda channels {auto-animate=true}

## Conda channels

Two common examples are:

- [bioconda]{.green} (a channel specializing in bioinformatics software)
- [conda-forge]{.green} (a community-led channel made up of thousands of contributors)

::: {data-id="m1"}
```{mermaid}
flowchart TD
ch1[(bioconda)] --- p1[bowtie2]
Expand All @@ -124,16 +112,14 @@ flowchart TD
ch2[(conda-forge)] --- p5[jupyter]
ch2[(conda-forge)] --- p6[wget]
```
:::

## Conda channels {auto-animate=true}
## Conda channels

Two common examples are:

- [bioconda]{.green} (a channel specializing in bioinformatics software)
- [conda-forge]{.green} (a community-led channel made up of thousands of contributors)

::: {data-id="m1"}
```{mermaid}
flowchart TD
ch1[(bioconda)] --- p1[bowtie2]
Expand All @@ -147,13 +133,12 @@ flowchart TD
p5 -.-> l1([conda install -c conda-forge -c bioconda snakemake jupyter])
p3 -.-> l1
```
:::

## Defining and sharing environments

Define a Conda environment in an `environment.yml` file:

``` {.yaml code-line-numbers="|1-3|4-12"}
```{.yaml filename="environment.yml" code-line-numbers="|1-3|4-12"}
channels:
- conda-forge
- bioconda
Expand All @@ -168,47 +153,48 @@ dependencies:
- graphviz=2.38.0
```

## Defining and sharing environments {auto-animate="true"}
## Conda command-line

- Update an existing environment:
::: {.fragment}
- Create a new environment:

```bash
conda env update -f environment.yml
conda env create -f environment.yml
```
:::

## Defining and sharing environments {auto-animate="true"}

::: {.fragment}
- Update an existing environment:

```bash
conda env update -f environment.yml
```
:::

- Export environment (including all dependencies) to a file:

```bash
conda env export > environment.yml
```

## Defining and sharing environments {auto-animate="true"}

::: {.fragment}
- Update an existing environment:

```bash
conda env update -f environment.yml
```
:::

::: {.fragment}

- Export environment (including all dependencies) to a file:

```bash
conda env export > environment.yml
```
:::

::: {.fragment}
- Export historical environment (only packages explicitly installed):

```bash
conda env export --from-history > environment.yml
```
:::

## Conda, Anaconda, Miniconda, Miniforge...

Expand Down
20 changes: 14 additions & 6 deletions lectures/containers/containers.qmd
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
---
title: 'Working with containers
title: 'Working with containers'
subtitle: '
<img src="images/docker-logo.webp" height="130px" style="float:center; padding: 15px 15px 15px 15px;"/>
<img src="https://apptainer.org/docs/user/main/_static/logo.png" height="130px" style="float:center; padding: 15px 15px 15px 15px;"/>
<img src="https://sylabs.io/guides/3.0/user-guide/_static/logo.png" height="130px" style="float:center; padding: 15px 15px 15px 15px;"/>'
'
format: revealjs
---

Expand Down Expand Up @@ -81,14 +82,20 @@ Status: Downloaded image for ubuntu:16.04
```
:::

::: {.fragment}
## An example: using a different OS

Check OS on local machine
```{.bash}
$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64
```

Run the container interactively and check OS version
```{.bash code-line-numbers="|1|2|3"}
$ docker run -it ubuntu:16.04
root@407b0fd13fe5:/## uname -a
Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux
```
:::

## Mounting volumes

Expand Down Expand Up @@ -168,9 +175,9 @@ docker run \
```
:::

## What is Singularity?
## What is Apptainer?

![](https://sylabs.io/guides/3.0/user-guide/_static/logo.png){height=250 fig-align=center}
![](http://apptainer.org/static/hero-img-e614806cf573fd0cbbe09a25d9e69637.png){fig-align=center}

::: {.incremental}
- Another software for working with containers, similar to Docker
Expand All @@ -183,6 +190,7 @@ docker run \
![](http://apptainer.org/static/hero-img-e614806cf573fd0cbbe09a25d9e69637.png){fig-align=center}

::: {.incremental}
- Apptainer used to be 'Singularity'.
- In 2021 Singularity joined the Linux Foundation and became
[Apptainer]{.green}.
- The company [Sylabs]{.green} still maintains Singularity as a commercial
Expand Down
68 changes: 16 additions & 52 deletions lectures/jupyter/jupyter.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -42,63 +42,47 @@ _The next-generation interface for Jupyter_
![](https://jupyterlab.readthedocs.io/en/stable/_images/jupyterlab.png){height="100%" fig-align=center}
::::
:::: {.column width="25%"}
::: {.incremental}
::: {.incremental .small}
- Similar to an integrated development environments (IDE).
- Tab views, Code consoles, Show output in a separate tab, Live rendering of edits _etc._
- The jupyter lab interface can run Jupyter notebooks in the **main** work area
:::
::::
:::::

## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" }
## Using Jupyter notebooks

::: {data-id="bullets"}
- Document your work in markdown...
:::

::: {data-id="markdown1"}
``` {.default}
**Penguin data analysis**
Here we will investigate the [Penguin dataset](https://github.com/allisonhorst/palmerpenguins).
---
The species included in this set are:
- _Adelie_
- _Chinstrap_
- _Gentoo_
```
:::
- _Adelie_
- _Chinstrap_
- _Gentoo_
```

## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" }
## Using Jupyter notebooks

::: {data-id="bullets"}
- Document your work in markdown...and have it rendered automatically.
:::

::: {data-id="markdown1"}

**Penguin data analysis**

Here we will investigate the [Penguin dataset](https://github.com/allisonhorst/palmerpenguins).

---

The species included in this set are:

- _Adelie_
- _Chinstrap_
- _Gentoo_
:::
- _Adelie_
- _Chinstrap_
- _Gentoo_

## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" }
## Using Jupyter notebooks

::: {data-id="bullets"}
- Document your work in markdown...and have it rendered automatically.
- Execute code directly from the browser, with results attached to the code which generated them.
:::

```{python}
#| echo: true
Expand All @@ -108,51 +92,31 @@ data = sns.load_dataset("penguins")
data.groupby("species").mean(numeric_only=True)
```

## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" }
## Using Jupyter notebooks

::: {data-id="bullets"}
- Document your work in markdown...and have it rendered automatically.
- Execute code directly from the browser, with results attached to the code which generated them.
- Mix and match languages in addition to `python`.
:::

::: {.fragment}
**R**
```{python}
#| echo: true
%%R
print(paste(Sys.Date(), ": Hello World", sep=""))
```

## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" }

::: {data-id="bullets"}
- Document your work in markdown...and have it rendered automatically.
- Execute code directly from the browser, with results attached to the code which generated them.
- Mix and match languages in addition to `python`.
:::

**R**
```{python}
#| echo: true
%%R
print(paste(Sys.Date(), ": Hello World", sep=""))
```

::: {.fragment}
**bash**
```{python}
#| echo: true
%%bash
echo "$(date): Hello World!"
```
:::

## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" }
## Using Jupyter notebooks

::: {data-id="bullets"}
- Document your work in markdown...and have it rendered automatically
- Execute code directly from the browser, with results attached to the code which generated them
- Mix and match languages in addition to `python`.
- Generate plots directly in the browser and/or save to file.
:::

```{python}
#| echo: true
Expand Down

0 comments on commit 97ab1b9

Please sign in to comment.