Db params #121

tkchafin · 2024-11-11T10:32:24Z

Addresses #110

Separated input databases from Sanger infrastructure in test cases
Pipeline now handles optional decompression of input databases

PR checklist

Dev

github-actions · 2024-11-11T10:34:22Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 7cfbcc0

+| ✅ 128 tests passed       |+
#| ❔  29 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: 0.7.0

❔ Tests ignored:

files_exist - File is ignored: CODE_OF_CONDUCT.md
files_exist - File is ignored: assets/nf-core-blobtoolkit_logo_light.png
files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_light.png
files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_dark.png
files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: conf/igenomes.config
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting_comment.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/email_template.txt
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File ignored due to lint config: assets/nf-core-blobtoolkit_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/blobtoolkit/blobtoolkit/.github/workflows/awstest.yml
template_strings - template_strings
merge_markers - merge_markers

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: lib/WorkflowBlobtoolkit.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-blobtoolkit_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: sanger_test_full.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: sanger_test.yml
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' contains report_section_order
multiqc_config - 'assets/multiqc_config.yml' contains export_plots
multiqc_config - 'assets/multiqc_config.yml' contains report_comment
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.11
Run at 2024-11-22 09:35:31

muffato

Also:

Delete the database files from assets/ since they're not needed any more.
Update the documentation
Update the CI workflow as it doesn't need to pre-download stuff anymore

conf/test_full.config

subworkflows/local/input_check.nf

Co-authored-by: Matthieu Muffato <[email protected]>

tkchafin · 2024-11-19T15:36:06Z

Also:

Delete the database files from assets/ since they're not needed any more.

Update the documentation

Update the CI workflow as it doesn't need to pre-download stuff anymore

all completed :)

muffato

Just added a comment about the database instructions not having to be followed entirely. I'm especially thinking of large databases like NT or Busco. There's no need to have the pipeline decompress them every time

docs/usage.md

Co-authored-by: Matthieu Muffato <[email protected]>

tkchafin added 3 commits October 14, 2024 18:47

database parameterisation and separate tests from sanger infrastructure

17b38ee

separate databases from sanger infra; handle compression

254fea1

Merge pull request #3 from tkchafin/dev

2afc63d

Dev

tkchafin linked an issue Nov 11, 2024 that may be closed by this pull request

Database parametrisation #110

Closed

tkchafin requested a review from muffato November 11, 2024 10:32

tkchafin self-assigned this Nov 11, 2024

muffato reviewed Nov 19, 2024

View reviewed changes

conf/test_full.config Outdated Show resolved Hide resolved

subworkflows/local/input_check.nf Outdated Show resolved Hide resolved

tkchafin and others added 6 commits November 19, 2024 09:42

Update conf/test_full.config

6399d0a

Co-authored-by: Matthieu Muffato <[email protected]>

Update subworkflows/local/input_check.nf

779ec64

Co-authored-by: Matthieu Muffato <[email protected]>

remove local database copies from assets

dddef77

remove pre-download from ci

a49bbd3

docs update

4e176e0

changelog update

959537d

tkchafin added 2 commits November 19, 2024 15:43

prettier linting

641d464

remove local files from CI

c5ab0a1

muffato approved these changes Nov 22, 2024

View reviewed changes

docs/usage.md Outdated Show resolved Hide resolved

Update docs/usage.md

7cfbcc0

Co-authored-by: Matthieu Muffato <[email protected]>

tkchafin merged commit 301bcff into sanger-tol:dev Nov 22, 2024
6 checks passed

tkchafin deleted the db_params branch November 22, 2024 09:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Db params #121

Db params #121

tkchafin commented Nov 11, 2024

github-actions bot commented Nov 11, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

muffato left a comment

tkchafin commented Nov 19, 2024

muffato left a comment

Db params #121

Db params #121

Conversation

tkchafin commented Nov 11, 2024

PR checklist

github-actions bot commented Nov 11, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

muffato left a comment

Choose a reason for hiding this comment

tkchafin commented Nov 19, 2024

muffato left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 11, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️