TheiaMeta: Viral Metagenomics workflow #64
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…form assembly with ivar, otherwise use shovil with megahit assembler
…d return the one with the highest base count
…and consensus assembly are generated concurrently, and the final contig is selected based on final assembly length (consensus len or total basepairs in aligned de novo contigs)
…apping. allow for the output of sam file format instead of the default PAF file.
…hese reads are now available under `read1_unmapped` and `read2_unmapped`
… directly, deprecating the bam2fastq task
* added helpful comments and changed read concatenation block to use samplenames that have hyphens instead of underscores. ran successfully with miniwdl, terra testing next * add frame work for GHA, working theiaprok_illumina_pe workflow * add placeholder filters, update filter check * avoid processing empty list * debug filtering * debug filtering * debug filtering * debug filtering * going back to original filter * add filter in single file * working prototype of updated qc_check * add tests for theiaprok_illumina_se, fix missing slash in shovill se task * add all theiaprok inputs to qc check task * remove ani species match from inputs, not used * add qc check the theiaprok_se * change workflows to use qc_check_phb task * fix task name * parse busco results * add qc_check_phb to theiaeuk * put busco results variable in quotes error * add qc check to theiaprok fasta and ont * update md5sums * add qc check to theiacov wfs * fix more md5sums * fix output file name * PHBG v1.3.0 changes - vibrio subworkflow * update description * expose min freq input for consensus and variant tasks * fix variable names * fix variable types * fix kraken empty string error * wdl doesn't have else * avoid empty string outputs * variable read as empty if equal to zero, enclose in quotes * update workflows to remove optional kraken outputs * update nor reproducible md5s * fix ci and local not matching * add quast to theiaprok fasta * add gc percent to theiaeuk outputs * add min_freq inputs to theiacov_illumina_se * add gc percent to theiaprok and theiaeuk qc check * add num reads to qc check theiaprok theiaeuk * recursion for assembly length check creates bug so removed * fix typo in pytest_filter.yml * add qc_check_phb task check to gha * update gha md5sums and qc check checks * typo corrected and fixed spacing on optional input * updated sra_fetch workflow to use fastq-dl v2.0.1. also exposed optional inputs for docker, disk_size, memory, cpus. tested fine with miniwdl * fix error on theiaprok * update checksums --------- Co-authored-by: kapsakcj <[email protected]> Co-authored-by: Robert A. Petit III <[email protected]> Co-authored-by: Sage Wright <[email protected]> Co-authored-by: Michelle Scribner <[email protected]> Co-authored-by: kevinlibuit <[email protected]> Co-authored-by: kevinlibuit <[email protected]>
* added helpful comments and changed read concatenation block to use samplenames that have hyphens instead of underscores. ran successfully with miniwdl, terra testing next * add frame work for GHA, working theiaprok_illumina_pe workflow * add placeholder filters, update filter check * avoid processing empty list * debug filtering * debug filtering * debug filtering * debug filtering * going back to original filter * add filter in single file * working prototype of updated qc_check * add tests for theiaprok_illumina_se, fix missing slash in shovill se task * add all theiaprok inputs to qc check task * remove ani species match from inputs, not used * add qc check the theiaprok_se * change workflows to use qc_check_phb task * fix task name * parse busco results * add qc_check_phb to theiaeuk * put busco results variable in quotes error * add qc check to theiaprok fasta and ont * update md5sums * add qc check to theiacov wfs * fix more md5sums * fix output file name * PHBG v1.3.0 changes - vibrio subworkflow * update description * expose min freq input for consensus and variant tasks * fix variable names * fix variable types * fix kraken empty string error * wdl doesn't have else * avoid empty string outputs * variable read as empty if equal to zero, enclose in quotes * update workflows to remove optional kraken outputs * update nor reproducible md5s * fix ci and local not matching * add quast to theiaprok fasta * add gc percent to theiaeuk outputs * add min_freq inputs to theiacov_illumina_se * add gc percent to theiaprok and theiaeuk qc check * add num reads to qc check theiaprok theiaeuk * recursion for assembly length check creates bug so removed * fix typo in pytest_filter.yml * add qc_check_phb task check to gha * update gha md5sums and qc check checks * typo corrected and fixed spacing on optional input * updated sra_fetch workflow to use fastq-dl v2.0.1. also exposed optional inputs for docker, disk_size, memory, cpus. tested fine with miniwdl * fix error on theiaprok * update checksums --------- Co-authored-by: kapsakcj <[email protected]> Co-authored-by: Robert A. Petit III <[email protected]> Co-authored-by: Sage Wright <[email protected]> Co-authored-by: Michelle Scribner <[email protected]> Co-authored-by: kevinlibuit <[email protected]> Co-authored-by: kevinlibuit <[email protected]>
|
TODO:
|
…ed and unmapped read files, as well as some assembly statistics regarding those files
andrewjpage
force-pushed
the
im-metagenomics-workflow
branch
from
a831131 to
99e9496
Compare
…cs into im-metagenomics-workflow
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #110
Setting as draft as development is still underway.🛠️ Changes Being Made
This PR features a new workflow, TheiaMeta_Illumina_PE, for the assembly of viral metagenomic data.
The diagram of the workflow is available below:
🧠 Context and Rationale
📋 Workflow/Task Steps
Please see the diagram above.
This workflow takes in Illumina PE data and performs:
The following quality metrics are computed:
Inputs
Mandatory inputs:
Optional outputs:
Outputs
{ "theiameta_illumina_pe.assembly_fasta": "${this.assembly_fasta}", "theiameta_illumina_pe.assembly_length": "${this.assembly_length}", "theiameta_illumina_pe.assembly_mean_coverage": "${this.assembly_mean_coverage}", "theiameta_illumina_pe.average_read_length": "${this.average_read_length}", "theiameta_illumina_pe.bbduk_docker": "${this.bbduk_docker}", "theiameta_illumina_pe.bedtools_docker": "${this.bedtools_docker}", "theiameta_illumina_pe.bedtools_version": "${this.bedtools_version}", "theiameta_illumina_pe.contig_number": "${this.contig_number}", "theiameta_illumina_pe.fastq_scan_docker": "${this.fastq_scan_docker}", "theiameta_illumina_pe.fastq_scan_version": "${this.fastq_scan_version}", "theiameta_illumina_pe.kraken2_docker": "${this.kraken2_docker}", "theiameta_illumina_pe.kraken2_percent_human_raw": "${this.kraken2_percent_human_raw}", "theiameta_illumina_pe.kraken2_percent_human_clean": "${this.kraken2_percent_human_clean}", "theiameta_illumina_pe.kraken2_report_raw": "${this.kraken2_report_raw}", "theiameta_illumina_pe.kraken2_report_clean": "${this.kraken2_report_clean}", "theiameta_illumina_pe.kraken2_version": "${this.kraken2_version}", "theiameta_illumina_pe.largest_contig": "${this.largest_contig}", "theiameta_illumina_pe.metaspades_docker": "${this.metaspades_docker}", "theiameta_illumina_pe.metaspades_version": "${this.metaspades_version}", "theiameta_illumina_pe.minimap2_docker": "${this.minimap2_docker}", "theiameta_illumina_pe.minimap2_version": "${this.minimap2_version}", "theiameta_illumina_pe.ncbi_scrub_docker": "${this.ncbi_scrub_docker}", "theiameta_illumina_pe.num_reads_clean1": "${this.num_reads_clean1}", "theiameta_illumina_pe.num_reads_clean2": "${this.num_reads_clean2}", "theiameta_illumina_pe.num_reads_clean_pairs": "${this.num_reads_clean_pairs}", "theiameta_illumina_pe.num_reads_raw1": "${this.num_reads_raw1}", "theiameta_illumina_pe.num_reads_raw2": "${this.num_reads_raw2}", "theiameta_illumina_pe.num_reads_raw_pairs": "${this.num_reads_raw_pairs}", "theiameta_illumina_pe.percent_coverage": "${this.percent_coverage}", "theiameta_illumina_pe.pilon_docker": "${this.pilon_docker}", "theiameta_illumina_pe.pilon_version": "${this.pilon_version}", "theiameta_illumina_pe.quast_docker": "${this.quast_docker}", "theiameta_illumina_pe.quast_version": "${this.quast_version}", "theiameta_illumina_pe.read1_clean": "${this.read1_clean}", "theiameta_illumina_pe.read1_dehosted": "${this.read1_dehosted}", "theiameta_illumina_pe.read1_mapped": "${this.read1_mapped}", "theiameta_illumina_pe.read1_unmapped": "${this.read1_unmapped}", "theiameta_illumina_pe.read2_clean": "${this.read2_clean}", "theiameta_illumina_pe.read2_dehosted": "${this.read2_dehosted}", "theiameta_illumina_pe.read2_mapped": "${this.read2_mapped}", "theiameta_illumina_pe.read2_unmapped": "${this.read2_unmapped}", "theiameta_illumina_pe.samtools_docker": "${this.samtools_docker}", "theiameta_illumina_pe.samtools_version": "${this.samtools_version}", "theiameta_illumina_pe.theiameta_illumina_pe_analysis_date": "${this.theiameta_illumina_pe_analysis_date}", "theiameta_illumina_pe.theiameta_illumina_pe_version": "${this.theiameta_illumina_pe_version}", "theiameta_illumina_pe.trimmomatic_docker": "${this.trimmomatic_docker}", "theiameta_illumina_pe.trimmomatic_version": "${this.trimmomatic_version}" }🧪 Testing
Locally
Tests passed locally with HAV blood sample with commit id 2d1a69b:
miniwdl run -v /home/ines_mendes/Git/public_health_bioinformatics/workflows/metagenomics/wf_theiameta_illumina_pe.wdl read1= ~/Test/HAV_Metagenomics/HAV0024_S8_L001_R1_001.fastq.gz read2= ~/Test/HAV_Metagenomics/HAV0024_S8_L001_R2_001.fastq.gz samplename=HAV0024 reference= ~/Test/HAV_Metagenomics/HAV.fastaTerra
commit id 2d1a69b
🔬 Quality checks
Pull Request (PR) checklist: