reduce unnecessary logging in MIDAS task #210
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…k lines of output. also made the 2 mv commands verbose (but it's only 2 lines!)
|
Resulting |
frankambrosio3
approved these changes
Oct 26, 2023
There was a problem hiding this comment.
Tested here:
https://app.terra.bio/#workspaces/theiagen-validations/ambrosio_validation_sandbox/job_history/8795c5a1-d76c-4088-a109-4e37b902036d
One job failure in that run was due to a failure in pasty task, and was not related to midas changes.
sage-wright
pushed a commit
that referenced
this pull request
Nov 13, 2023
* updated VCF output file renaming in kSNP3 task (#207) * updated VCF output file renaming in kSNP3 task; also added 1 new File output and change the output names to be more descriptive * ksnp3 task:changed VCF file names to be predictable; split 2 ksnp3 options to 2 lines for readability; added new string output "ksnp3_vcf_ref_samplename" to capture sample within cluster to use for snp calling * added new string output to ksnp3 workflow "ksnp3_vcf_ref_samplename" * reduce unnecessary logging in MIDAS task (#210) * made untar/decompression of midas database quiet since it produces 41k lines of output. also made the 2 mv commands verbose (but it's only 2 lines!) * update CI * expose tbprofiler parameters as inputs in merlin * input spelling --------- Co-authored-by: Curtis Kapsak <[email protected]>
kevinlibuit
added a commit
that referenced
this pull request
Dec 29, 2023
…245) * output tbprofiler vcf * update default docker * fix path * add sample id to the beginning of the coverage report * update default docker * Enable TBProfiler parameter changes (#246) * updated VCF output file renaming in kSNP3 task (#207) * updated VCF output file renaming in kSNP3 task; also added 1 new File output and change the output names to be more descriptive * ksnp3 task:changed VCF file names to be predictable; split 2 ksnp3 options to 2 lines for readability; added new string output "ksnp3_vcf_ref_samplename" to capture sample within cluster to use for snp calling * added new string output to ksnp3 workflow "ksnp3_vcf_ref_samplename" * reduce unnecessary logging in MIDAS task (#210) * made untar/decompression of midas database quiet since it produces 41k lines of output. also made the 2 mv commands verbose (but it's only 2 lines!) * update CI * expose tbprofiler parameters as inputs in merlin * input spelling --------- Co-authored-by: Curtis Kapsak <[email protected]> * update md5sums * caller_options tbprofiler * caller_options merlin magic * --calling_params tbprofiler * calling_params tbprofiler * quotes around params tbprofiler * added quotes around calling params tbprofiler * "-C 1 -F 0.0" tbprof * removed caller options * hardcoded tbprofiler freebayes params * re-optionalize * update md5sums * Add branch name to versioning task * version reversion for merge * update checksums --------- Co-authored-by: frankambrosio3 <[email protected]> Co-authored-by: Curtis Kapsak <[email protected]> Co-authored-by: frankambrosio3 <[email protected]> Co-authored-by: kevinlibuit <[email protected]>
cimendes
pushed a commit
that referenced
this pull request
Apr 15, 2024
* output tbprofiler vcf * update default docker * fix path * add sample id to the beginning of the coverage report * update default docker * Enable TBProfiler parameter changes (#246) * updated VCF output file renaming in kSNP3 task (#207) * updated VCF output file renaming in kSNP3 task; also added 1 new File output and change the output names to be more descriptive * ksnp3 task:changed VCF file names to be predictable; split 2 ksnp3 options to 2 lines for readability; added new string output "ksnp3_vcf_ref_samplename" to capture sample within cluster to use for snp calling * added new string output to ksnp3 workflow "ksnp3_vcf_ref_samplename" * reduce unnecessary logging in MIDAS task (#210) * made untar/decompression of midas database quiet since it produces 41k lines of output. also made the 2 mv commands verbose (but it's only 2 lines!) * update CI * expose tbprofiler parameters as inputs in merlin * input spelling --------- Co-authored-by: Curtis Kapsak <[email protected]> * update md5sums * caller_options tbprofiler * caller_options merlin magic * --calling_params tbprofiler * calling_params tbprofiler * quotes around params tbprofiler * added quotes around calling params tbprofiler * "-C 1 -F 0.0" tbprof * removed caller options * hardcoded tbprofiler freebayes params * re-optionalize * update md5sums * draft tbprofiler tngs * add versioning * add to dockstore * commenting out clockwork to try and fix bugs? * chop 30 bp from both sides * update workflow to use trimmomatic chop * remove whitespace cruft * merge into regular trimmomatic task * change naming * prevent widespread failures * update to latest version of tbp_parser and enable tngs bed file * update md5sum * tngs updates * update docker * update docker * udpate paths * update docker * enable rrs & rrl frequency changeing * update docker * update md5sum * write all tbprofiler outputs * remove comment cruft * remove database for tbprofiler; broken * fix issue * update version and allow for user-modifable expert rule regions bed file * add rpob & etha freq modification params * add new parameters for modification * optionalize * v1.3.9 * new version!!!!!!1 * update md5sums --------- Co-authored-by: frankambrosio3 <[email protected]> Co-authored-by: Curtis Kapsak <[email protected]> Co-authored-by: frankambrosio3 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🛠️ Changes Being Made
made the un-tar/decompression of midas database (.tar.gz file) quiet since it produces 41k lines of output.
When using the
tar -vflag with this database, it results in a log file that is 2.7MB in size 😱also made the 2
mvcommands verbose (but it's only 2 lines!)🧠 Context and Rationale
lower storage cost, and task should run faster since it doesn't have to produce the 41k lines in the STDOUT
📋 Workflow/Task Steps
N/A
Inputs
N/A
Outputs
🧪 Testing
Locally
planning to only test in Terra since it's a simple code change
Terra
Test here
(still running as of 2023-10-06 afternoon) but I expect it to succeedwhich ran successfully: https://app.terra.bio/#workspaces/theiagen-validations/curtis-sandbox-theiagen-validations/job_history/41fe3b0b-f92f-4f25-bcc0-d894090e2efe🔬 Quality checks
Pull Request (PR) checklist: