Ingest with nextclade #62
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joverlee521
added a commit
that referenced
this pull request
Jun 24, 2024
Motivated by my own need to test the ingest workflows for the latest addition of Nextclade outputs in #62.
3 tasks
joverlee521
force-pushed
the
ingest-with-nextclade
branch
from
58df478 to
f33157f
Compare
Using `community/moncla-lab/iav-h5/ha/all-clades` as the default Nextclade dataset since it works across fauna and NCBI data. Subsequent commits will join these rules with the full ingest workflows.
Using the nextclade_field_map that's currently used in the measles ingest workflow.¹ We can cut down on the columns used if they are not useful for avian flu. ¹ <https://github.com/nextstrain/measles/blob/957fc744c64b8f5a722b5c525687d0746755add6/ingest/defaults/nextclade_field_map.tsv>
We are not using the alignment.fasta anywhere and I don't think it makes sense to only upload alignment for the HA segment.
Keep a copy of the full Nextclade TSV output from ingest on S3 since we won't necessarily join all columns with the metadata output.
joverlee521
force-pushed
the
ingest-with-nextclade
branch
from
f33157f to
8adda40
Compare
joverlee521
commented
Jun 24, 2024
Comment on lines
+8
to
+28
| coverage coverage | ||
| totalMissing missing_data | ||
| totalSubstitutions divergence | ||
| totalNonACGTNs nonACGTN | ||
| qc.overallStatus QC_overall | ||
| qc.missingData.status QC_missing_data | ||
| qc.mixedSites.status QC_mixed_sites | ||
| qc.privateMutations.status QC_rare_mutations | ||
| qc.snpClusters.status QC_snp_clusters | ||
| qc.frameShifts.status QC_frame_shifts | ||
| qc.stopCodons.status QC_stop_codons | ||
| frameShifts frame_shifts | ||
| privateNucMutations.reversionSubstitutions private_reversion_substitutions | ||
| privateNucMutations.labeledSubstitutions private_labeled_substitutions | ||
| privateNucMutations.unlabeledSubstitutions private_unlabeled_substitutions | ||
| privateNucMutations.totalReversionSubstitutions private_total_reversion_substitutions | ||
| privateNucMutations.totalLabeledSubstitutions private_total_labeled_substitutions | ||
| privateNucMutations.totalUnlabeledSubstitutions private_total_unlabeled_substitutions | ||
| privateNucMutations.totalPrivateSubstitutions private_total_private_substitutions | ||
| qc.snpClusters.clusteredSNPs private_snp_clusters | ||
| qc.snpClusters.totalSNPs private_total_snp_clusters |
There was a problem hiding this comment.
Should we only keep the clade assignment and drop all of these other columns? These QC outputs are specific to the HA segment so it might not make sense to keep as part of the overall metadata.
genehack
approved these changes
Jun 24, 2024
joverlee521
commented
Jun 24, 2024
| # Nextclade can have pathogen specific output columns so make sure to check which | ||
| # columns would be useful for your downstream phylogenetic analysis. | ||
| seqName seqName | ||
| clade clade |
There was a problem hiding this comment.
Would it make sense to include the polybasic_cleavage_site output column or should the phylogenetic builds continue to rely on scripts/annotate-ha-cleavage-site.py?
joverlee521
commented
Jun 24, 2024
Comment on lines
+68
to
+72
| { | ||
| "key": "clade", | ||
| "title": "Nextclade Clade", | ||
| "type": "categorical" | ||
| }, |
There was a problem hiding this comment.
I've added the Nextclade Clade as a separate coloring so we can do comparisons across clade labels, but maybe we'll remove h5_label_clade eventually? Would love to hear your thoughts here @lmoncla 🙏
genehack
approved these changes
Jun 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of proposed changes
Runs Nextclade as part of the ingest workflow so that we get Nextclade clade annotations for all H5 HA sequences.
Uses the
community/moncla-lab/iav-h5/ha/all-cladesNextclade dataset.Related issue(s)
Resolves #44
Checklist
-> results at
s3://nextstrain-data-private/files/workflows/avian-flu/trial/ingest-with-nextclade/metadata.tsv.zsts3://nextstrain-data/files/workflows/avian-flu/trial/ingest-with-nextclade/h5n1/metadata.tsv.zst