Refactoring diseases parsing files

[northwestwitch]

Let's have a discussion here on how to include ORPHANET terms into Scout. Since where we parse the terms from,how we collect associated genes and HPO phenotypes is quite confusing (at least for me), let's do a recap of what we want to achieve here.

Current situation:

• Only OMIM terms are included into Scout.
• OMIM terms are parsed from the file named genemap2, which is downloaded from OMIM with an api key and contains OMIM terms associated to genes.(populates the database with roughly 6781 terms).
• The phenotype.hpoa file is also downloaded and is used to associate HPO terms to the OMIM terms parsed above.
• Genes associated to OMIM terms come from the genemap2 file.

Proposed change:

Assuming that phenotype.hpoa contains all disease terms (Both OMIM & ORPHA) - contains also Decipher but we leave it out for now:

• Start parsing all terms from phenotype.hpoa
• Collect phenotype terms associated to diseases also from phenotype.hpoa
• Collect genes associated to diseases using the phenotype_to_genes.txt file, given that we already have the HPO terms associated to the diseases.

Before doing so we should check that:

• All OMIM terms that we are currently parsing are included in phenotype.hpoa
• All genes associated to the OMIM terms and that we currently collect from genemap2 would be the same when collected using the phenotype_to_genes.txt file.

I still feel that I don't have a complete understanding of this, and that's why I've started this discussion. Any ideas - stuff I'm missing?

[northwestwitch]

@TereseBo @dnil

[dnil]

I'll try to have a look late afternoon, or possibly have to defer it to next week, but my birdseye for now is to be careful to separate:

1. mapping phenotype to genes (application critical, must not disrupt, but improvements are of course welcome). From the top of my head, we rely on HPO (JAX) for this mapping. Not expected to change with an addition of ORPHA.
2. diseases to gene mapping. inheritance modes per gene etc. Also critical, but somewhat less so. Very appreciated by analysts. Derived from OMIM. It would be nice if we could get gene to ORPHA mapping, but thats a bonus.
3. phenotype listing - HPO derived, not expected to change.
4. disease catalog (for use in eg assigning diagnoses, submitting to matchmaker and clinvar, reporting causatives). Low hanging fruit for including ORPHA!
5. disease to phenotype mapping - afaik relatively non-critical to the application, more of an information point. I get fuzzy about how that is sourced currently without reading PRs from the most recent changes, but we used to take a union of HPO and OMIM info here. I should think we will get an addition here just parsing up the ORPHA terms from the same HPO file we get the OMIM ones from.

[northwestwitch]

Ah yes, gene inheritance models should be preserved, of course!

[TereseBo]

I have poked around some more: I now load 13234 diseases to the database. I think the added 813 codes were present in the genmap2 but not in the phenotype.hpoa which includes 8181: OMIM terms and 4242 ORPHA terms according to the file-comments

[northwestwitch]

So it seems that in order OMIM terms you'd need both genemap2 and phenotype.hpoa, because phenotype.hpoa contains only terms with at least one associated HPO term.

So probably we would need to fetch the ORPHA terms also from somewhere else. From here perhaps?