Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework model/batch size configuration again #14

Open
wants to merge 30 commits into
base: dev
Choose a base branch
from

Conversation

jeremybobbin
Copy link
Contributor

  • If GPU_INDICIES contains GPUs that are of different models, the script exits. This assumption makes (future)logging simpler.
  • Batch size is calculated based on a model's batch size multiplier(seen below), precision, and GPU memory:
  resnet50='5  + 1/3'
 resnet152='2  + 2/3'
inception3='5  + 1/3'
inception4='1  + 1/3'
     vgg16='5  + 1/3'
   alexnet='42 + 2/3'
    ssd300='2  + 2/3'
  • benchmark.sh's options are now position independent:
./benchmark.sh -l 2 -h 4     # low/high - benchmarks 2 GPUs, 3 GPUs and 4GPUs at a time.

Options:

i - GPU_INDEX
l - MIN_NUM_GPU
h - MAX_NUM_GPU
n - ITERATIONS
b - NUM_BATCHES
c - SETTING
v - GPU_VENDOR
t - THERMAL_INTERVAL
  • -n has implemented the functionality of batch_benchmark.sh

This:

./batch_benchmark.sh 1 1 1 100 2 config_resnet50_replicated_fp32_train_syn

Is now this:

./benchmark.sh -h 1 -n 1 -b 100 -t 2 -c config_resnet50_replicated_fp32_train_syn

jeremybobbin and others added 30 commits July 4, 2020 11:11
Create tf2 branch. Add official tensorflow benchmark repo as a sub-module

checkout branch for tf1.15
I'm skeptical that we'll ever want to use heterogeneous GPU
configurations
This patch renders batch_benchmark.sh redundent.
@jeremybobbin
Copy link
Contributor Author

jeremybobbin commented Jul 17, 2020

Issues:

  • requires bc, fails mysteriously otherwise
  • CPU_NAME is not set

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants