ingest: simplify NCBI Datasets fields config #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of hard-coding the list of NCBI Datasets fields in the workflow, just provide the list via the default config. This makes it easy to customize which fields to include and makes it very obvious that field_map config for the the curation pipeline is changing the names of these NCBI fields. This includes a change in the `format_ncbi_dataset_report` rule to use the provided fields as the header so that we do not have to do a separate renaming of the NCBI column names back to the computer friendly mneumonics.
joverlee521
commented
Nov 28, 2023
Comment on lines
+48
to
+63
| accession: accession | ||
| sourcedb: database | ||
| sra-accs: sra_accessions | ||
| isolate-lineage: strain | ||
| geo-region: region | ||
| geo-location: location | ||
| isolate-collection-date: date | ||
| release-date: date_released | ||
| update-date: date_updated | ||
| length: length | ||
| host-name: host | ||
| isolate-lineage-source: sample_type | ||
| biosample-acc: biosample_accessions | ||
| submitter-names: authors | ||
| submitter-affiliation: institution | ||
| submitter-country: submitter_country |
There was a problem hiding this comment.
This will most likely change in the near future when we've resolved #20
There was a problem hiding this comment.
[non blocking comment]
I lean toward changing accession: accession to accession: genbank_accession to be specific for NCBI data. Therefore, we can define downstream merging of NCBI and other (or private) data to generate a harmonized accession record ID.
j23414
approved these changes
Nov 28, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of proposed changes
Instead of hard-coding the list of NCBI Datasets fields in the workflow, just provide the list via the default config. This makes it easy to customize which fields to include and makes it very obvious that field_map config for the the curation pipeline is changing the names of these NCBI fields.
This includes a change in the
format_ncbi_dataset_reportrule to use the provided fields as the header so that we do not have to do a separate renaming of the NCBI column names back to the computer friendly mneumonics.Related issue(s)
Prompted by my review of dengue ingest PR nextstrain/dengue#13 (comment)
Checklist