Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds batching support to run predictions in larger batches to get better gpu utilization and speed. See comments in readme for speed comparisons.
--prediction-batch-size
and--tensorflow-batch-size
to support batching variants to optimize prediction utilization on a GPUVCFPredictionBatch
class that manages collection the VCF records, placing them in batches based on the encoded tensor size. Once the batch size is reached, predictions are run in batches, then output is written back in the original order reassembling the annotations for the VCF record. Each VCF record has a lookup key for where each of the ref/alts are within their batches, so it knows where to grab the results during reassemblyget_delta_scores
method into reusable methods used in the batching and the original source code. This way the batching code can utilize the same logic inside that method while still maintaining the original versionget_delta_scores
.encode_batch_record
handles what was in the first half, taking in the VCF record and generating one-hot encoded matrices for the ref/alts.extract_delta_scores
handles the second half of theget_delta_scores
by reassembling the annotations based on the batched predictions