Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for using cross-organisational components #2816

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,9 @@ There are four additional flags that you can use when installing a subworkflow:
- `--force`: Overwrite a previously installed version of the subworkflow.
- `--prompt`: Select the subworkflow version using a cli prompt.
- `--sha <commit_sha>`: Install the subworkflow at a specific commit.

:::note
You can use the `meta.yml` file to specify different remotes for the components
in your subworkflow. This is useful when you want to use modules from organisations
different from the one where the subworkflow is located. For more information, [see our tutorial](/docs/tutorials/external_usage/cross_organisational_subworkflows).
:::
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: 'Using subworkflows with modules from multiple organisations'
subtitle: Guidance on how to use nf-core/tools with subworkflows that use modules from more than one organisation.
---

In order to use cross-organisational subworkflows, you can specify individual remotes to be used for each module within the `meta.yml`
file in that subworkflow.

For example, let's say you have a subworkflow in your own organisation's `modules` repository that uses FastQC and fastp.
But, instead of using the fastp module from your own organisation, you want to use nf-core's fastp module. This is the case for the `fastq_trim_fastp_fastqc` subworkflow in the [nf-core-test](https://github.com/nf-core-test/modules/tree/main/subworkflows/nf-core-test/fastq_trim_fastp_fastqc) organization.

In order to have this subworkflow, with a FastQC module from your organisation and fastp from nf-core, you'd define the
components section of this subworkflow's `meta.yml` file as such:

```yaml title="meta.yml" {11-12}
name: 'fastq_trim_fastp_fastqc'
description: Read QC, fastp trimming and read qc
keywords:
- qc
- quality_control
- adapters
- trimming
- fastq
components:
- fastqc
- fastp:
git_remote: https://github.com/nf-core/modules.git
```

By using the structure above, `nf-core/tools` knows that it should install FastQC from the same repository that the subworkflow is located
in, but fastp should come from the `nf-core/modules` remote.

:::note
Be aware that to install subworkflows from organisations other than nf-core you must
specify `--git-remote` in the `nf-core subworkflows` command, e.g. `nf-core subworkflows install fastq_trim_fastp_fastqc --git-remote $ORG_NAME{:bash}`
:::

If you're using cross-organisational subworkflows in your repository, be aware that you must also specify a different JSON schema
to lint the `meta.yml` files for your subworkflows.
To do this, just change the `components` section type to allow both objects and strings.

```json title="yaml-schema.json" {5}
"components": {
"type": "array",
"description": "Modules and subworkflows used in the subworkflow",
"items": {
"type": ["object", "string"]
},
"minItems": 0
},
```

:::tip
To get the full content for the schema, see the corresponding
file in the [nf-core-test](https://github.com/nf-core-test/modules/blob/main/subworkflows/yaml-schema.json) modules repository.
:::
Loading