KmerFinder to TheiaProk

[cimendes]

Closes #187

🛠️ Changes Being Made

This PR adds the option to run Kmerfinder for bacterial genomes if the user requests it.

Bonus: sistr's sistr_allele_fasta no longer reads sister_allele_fasta (@emmadoughty)

🧠 Context and Rationale

• The kmerfinder bacterial database was uploaded to RP Google bucket, available at gs://theiagen-public-files-rp/terra/theiaprok-files/kmerfinder_bacteria.tar.gz. It's about 30Gb in size.
• KmerFinder_bacterial task was implemented
• a Boolean was added in TheiaProk_illumina_PE, TheiaProk_illumina_SE, TheiaProk_Fasta and TheiaProk_ONT to optionally run Kmerfinder

📋 Workflow/Task Steps

• The database is decompressed and passed to kmerfinder
• The resulting txt file is parsed to extract the species ID of the top hit
• the results file and top hit are outputted

Inputs

• New input call_kmerfinder set to false by default

Outputs

output {
    String kmerfinder_docker = docker
    File? kmerfinder_results_tsv = "~{samplename}_kmerfinder.tsv"
    String kmerfinder_top_hit = read_string("TOP_HIT")
    String kmerfinder_query_coverage = read_string("QC_METRIC")
    String kmerfinder_template_coverage = read_string("TEMPLATE_COVERAGE")
    String kmerfinder_database = read_string("DATABASE")
  }

🧪 Testing

Locally

Tested task locally with miniwdl

Terra

Underway!!!

TheiaProk_Illumina_PE (n=55): https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/349c9d0f-b2b6-4e94-8156-7680a372a7b6

TheiaProk_Illumina_SE (n=55): https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/47a60d7f-e33b-4cda-97d3-2bb5fd22917b

TheiaProk_FASTA (n=55): https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/ef0e66e5-9d3f-4199-bde3-807b1db7e2f2

Table of differences between Gambit and KmerFinder:
amrfinderplus_testing_sample.zip

TheiaProk_ONT (n=44): https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Mendes_Sandbox/job_history/6f46335b-1d52-490b-b289-ee274a4fbc86

🔬 Quality checks

Pull Request (PR) checklist:

• Include a description of what is in this pull request in this message.
• The workflow/task has been tested locally and on Terra
• The CI/CD has been adjusted and tests are passing
• Everything follows the style guide

[michellescribner]

Launched test of TheiaProk_FASTA_PHB: https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Scribner_Sandbox/job_history/3b03f4db-7036-4289-a4cc-020b22c89f82

[michellescribner]

Thanks for all of the changes @cimendes! I just launched a test of TheiaProk_Illumina_PE_PHB https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Scribner_Sandbox/job_history/f644e3cd-a563-473f-b9d3-9e9f4c0fbc8b

[michellescribner]

Bug: If kmerfinder does not make any prediction (the result file is empty), the task will populate the string output with the header line. See example sample in this run: https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Scribner_Sandbox/job_history/f644e3cd-a563-473f-b9d3-9e9f4c0fbc8b.

If there is no output from Kmerfinder, the ideal outcome is for these strings to be populated with "No match detected" or similar.

[cimendes]

Thanks @michellescribner for finding this nasty bug! I've added a patch that fixes this scenario and the headers no longer are populated in the datatable. Instead the "No hit detected in database" string appears!

[michellescribner]

Thank you @cimendes for making this change so quickly! I am relaunching my tests for:
TheiaProk_Illumina_PE_PHB, n=116: https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Scribner_Sandbox/job_history/2e2dd0d6-8a1f-491b-a3c5-a3af3fc15811
TheiaProk_FASTA_PHB, n=15: https://app.terra.bio/#workspaces/theiagen-validations/Theiagen_Scribner_Sandbox/job_history/bf439e88-4b52-4366-a7a7-bc3624552fe4

[michellescribner]

Tested by @kelseykropp and found that 27/27 samples showed results that matched the kmerfinder online tool. https://app.terra.bio/#workspaces/theiagen-training-workspaces/Theiagen_Kropp_Sandbox/job_history/be7a6dca-b5b0-4376-92a9-a827bbbc1da8
One sample failed this analysis due to a workflow error at the ANI step. Seems to be an independent bug in the ANI module.

[michellescribner]

Great job @cimendes!