Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabled dataset to be used from host - resnet50 and retinanet #152

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3db5cd2
used copy instead of softlink in retinanet
anandhu-eng Oct 13, 2024
93c4507
enabled retinanet model download
anandhu-eng Oct 14, 2024
f439d78
reverted dataset path changes
anandhu-eng Oct 15, 2024
fe750b5
reverted retinanet model download
anandhu-eng Oct 15, 2024
9802874
Merge branch 'mlperf-inference' into sdxl_accuracy_fixes
anandhu-eng Oct 15, 2024
900f459
added env key - CM_OPENIMAGES_CALIBRATION_DATASET_PATH
anandhu-eng Oct 15, 2024
86ea53a
Updated new env keys
anandhu-eng Oct 15, 2024
210262f
added env key - CM_DATASET_OPENIMAGES_VALIDATION_ANNOTATIONS_FILE_PATH
anandhu-eng Oct 15, 2024
90f7528
updated new env keys
anandhu-eng Oct 15, 2024
3acbced
download retinanet dataset in host
anandhu-eng Oct 15, 2024
ab6afca
disabled model download inside docker - nvidia
anandhu-eng Oct 15, 2024
e3ee8a8
updated env variables for annotation and data
anandhu-eng Oct 15, 2024
0f9c48e
modifed env variable for openimage dataset
anandhu-eng Oct 15, 2024
94aadf0
added CM_DATASET_OPENIMAGES_PATH to mount
anandhu-eng Oct 15, 2024
926deae
updated new env keys
anandhu-eng Oct 16, 2024
2c1aba3
added skip if env condition for dataset download
anandhu-eng Oct 16, 2024
46d62e8
added common variable for using dataset from host
anandhu-eng Oct 16, 2024
4a3decb
added use_dataset_from_host input mapping
anandhu-eng Oct 16, 2024
815695d
Merge branch 'mlperf-inference' into sdxl_accuracy_fixes
anandhu-eng Oct 16, 2024
49a8d7d
fix typo
anandhu-eng Oct 16, 2024
b519cfd
skipped dataset download while building container
anandhu-eng Oct 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions script/app-mlperf-inference-nvidia/_cm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,11 @@ deps:
- enable_if_env:
CM_MODEL:
- resnet50
skip_if_env:
CM_USE_DATASET_FROM_HOST:
- 'yes'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add a condition to skip this only during the docker build stage - I think it is CM_RUN_STATE_DOCKER=yes

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated: b519cfd

CM_RUN_STATE_DOCKER:
- 'yes'
names:
- imagenet-original
tags: get,dataset,original,imagenet,_full
Expand Down Expand Up @@ -218,13 +223,23 @@ deps:
- enable_if_env:
CM_MODEL:
- retinanet
skip_if_env:
CM_USE_DATASET_FROM_HOST:
- 'yes'
CM_RUN_STATE_DOCKER:
- 'yes'
names:
- openimages-original
tags: get,dataset,original,openimages,_validation,_full,_custom-annotations

- enable_if_env:
CM_MODEL:
- retinanet
skip_if_env:
CM_USE_DATASET_FROM_HOST:
- 'yes'
CM_RUN_STATE_DOCKER:
- 'yes'
names:
- openimages-calibration
tags: get,dataset,original,openimages,_calibration
Expand Down
6 changes: 3 additions & 3 deletions script/app-mlperf-inference-nvidia/customize.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,10 +138,10 @@ def preprocess(i):

elif env['CM_MODEL'] == "retinanet":
#print(env)
dataset_path = env['CM_DATASET_PATH']
dataset_path = env['CM_DATASET_OPENIMAGES_PATH']
#return {'return': 1, 'error': 'error'}

annotations_path = env['CM_DATASET_ANNOTATIONS_DIR_PATH']
annotations_path = env['CM_DATASET_OPENIMAGES_VALIDATION_ANNOTATIONS_FILE_PATH']
target_data_path_dir = os.path.join(env['MLPERF_SCRATCH_PATH'], 'data', 'open-images-v6-mlperf')
if not os.path.exists(target_data_path_dir):
cmds.append(f"mkdir -p {target_data_path_dir}")
Expand All @@ -156,7 +156,7 @@ def preprocess(i):
if not os.path.exists(target_data_path):
cmds.append(f"ln -sf {dataset_path} {target_data_path}")

calibration_dataset_path=env['CM_CALIBRATION_DATASET_PATH']
calibration_dataset_path=env['CM_OPENIMAGES_CALIBRATION_DATASET_PATH']
target_data_path_dir = os.path.join(env['MLPERF_SCRATCH_PATH'], 'data', 'open-images-v6-mlperf','calibration', 'train')
if not os.path.exists(target_data_path_dir):
cmds.append(f"mkdir -p {target_data_path_dir}")
Expand Down
24 changes: 23 additions & 1 deletion script/app-mlperf-inference/_cm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ input_mapping:
gpu_name: CM_NVIDIA_GPU_NAME
nvidia_llama2_dataset_file_path: CM_NVIDIA_LLAMA_DATASET_FILE_PATH
tp_size: CM_NVIDIA_TP_SIZE
use_dataset_from_host: CM_USE_DATASET_FROM_HOST

# Duplicate CM environment variables to the ones used in native apps
env_key_mappings:
Expand Down Expand Up @@ -596,10 +597,12 @@ variations:
- mlperf-accuracy-script
- imagenet-accuracy-script
tags: run,accuracy,mlperf,_imagenet

docker:
deps:
- tags: get,dataset,imagenet,validation,original,_full
enable_if_env:
CM_USE_DATASET_FROM_HOST:
- 'yes'
names:
- imagenet-original
- dataset-original
Expand All @@ -625,6 +628,22 @@ variations:
- openimages-accuracy-script
tags: run,accuracy,mlperf,_openimages

retinanet,nvidia-original:
docker:
deps:
- names:
- openimages-original
enable_if_env:
CM_USE_DATASET_FROM_HOST:
- 'yes'
tags: get,dataset,original,openimages,_validation,_full,_custom-annotations
- names:
- openimages-calibration
enable_if_env:
CM_USE_DATASET_FROM_HOST:
- 'yes'
tags: get,dataset,original,openimages,_calibration

3d-unet-99:
group:
model
Expand Down Expand Up @@ -1636,6 +1655,9 @@ docker:
- cm pull repo
mounts:
- "${{ CM_DATASET_IMAGENET_PATH }}:${{ CM_DATASET_IMAGENET_PATH }}"
- "${{ CM_DATASET_OPENIMAGES_PATH }}:${{ CM_DATASET_OPENIMAGES_PATH }}"
- "${{ CM_OPENIMAGES_CALIBRATION_DATASET_PATH }}:${{ CM_OPENIMAGES_CALIBRATION_DATASET_PATH }}"
- "${{ CM_DATASET_OPENIMAGES_VALIDATION_ANNOTATIONS_FILE_PATH }}:${{ CM_DATASET_OPENIMAGES_VALIDATION_ANNOTATIONS_FILE_PATH }}"
- "${{ CM_MLPERF_INFERENCE_RESULTS_DIR }}:${{ CM_MLPERF_INFERENCE_RESULTS_DIR }}"
- "${{ OUTPUT_BASE_DIR }}:${{ OUTPUT_BASE_DIR }}"
- "${{ CM_MLPERF_INFERENCE_SUBMISSION_DIR }}:${{ CM_MLPERF_INFERENCE_SUBMISSION_DIR }}"
Expand Down
5 changes: 4 additions & 1 deletion script/get-dataset-openimages/_cm.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,14 +69,17 @@
"new_env_keys": [
"CM_DATASET_PATH",
"CM_DATASET_PATH_ROOT",
"CM_DATASET_OPENIMAGES_PATH",
"CM_DATASET_OPENIMAGES_DATASET_PATH",
"CM_DATASET_OPENIMAGES_DATASET_PATH_ROOT",
"CM_DATASET_ANNOTATIONS_DIR_PATH",
"CM_DATASET_ANNOTATIONS_FILE_PATH",
"CM_DATASET_CALIBRATION_ANNOTATIONS_FILE_PATH",
"CM_DATASET_VALIDATION_ANNOTATIONS_FILE_PATH",
"CM_CALIBRATION_DATASET_PATH",
"CM_CALIBRATION_DATASET_PATH_ROOT"
"CM_CALIBRATION_DATASET_PATH_ROOT",
"CM_OPENIMAGES_CALIBRATION_DATASET_PATH",
"CM_DATASET_OPENIMAGES_VALIDATION_ANNOTATIONS_FILE_PATH"
],
"tags": [
"get",
Expand Down
2 changes: 2 additions & 0 deletions script/get-dataset-openimages/customize.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ def postprocess(i):
env['CM_DATASET_PATH'] = os.path.join(os.getcwd(), 'install', 'validation', 'data')
annotations_file_path = os.path.join(env['CM_DATASET_ANNOTATIONS_DIR_PATH'], "openimages-mlperf.json")
env['CM_DATASET_VALIDATION_ANNOTATIONS_FILE_PATH'] = annotations_file_path
env['CM_DATASET_OPENIMAGES_VALIDATION_ANNOTATIONS_FILE_PATH'] = annotations_file_path
env['CM_DATASET_ANNOTATIONS_FILE_PATH'] = annotations_file_path
if env.get("CM_DATASET_OPENIMAGES_CUSTOM_ANNOTATIONS",'') == "yes":
annotations_file_src = env['CM_DATASET_OPENIMAGES_ANNOTATIONS_FILE_PATH']
Expand All @@ -77,6 +78,7 @@ def postprocess(i):
env['CM_DATASET_OPENIMAGES_PATH_ROOT'] = env['CM_DATASET_PATH_ROOT']
else:
env['CM_CALIBRATION_DATASET_PATH'] = os.path.join(os.getcwd(), 'install', 'calibration', 'data')
env['CM_OPENIMAGES_CALIBRATION_DATASET_PATH'] = os.path.join(os.getcwd(), 'install', 'calibration', 'data')
env['CM_CALIBRATION_DATASET_PATH_ROOT'] = os.path.join(os.getcwd(), 'install')
annotations_file_path = os.path.join(env['CM_DATASET_ANNOTATIONS_DIR_PATH'], "openimages-calibration-mlperf.json")
env['CM_DATASET_CALIBRATION_ANNOTATIONS_FILE_PATH'] = annotations_file_path
Expand Down
1 change: 1 addition & 0 deletions script/run-mlperf-inference-app/_cm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ input_mapping:
max_test_duration: CM_MLPERF_MAX_DURATION_TEST
all_models: CM_MLPERF_ALL_MODELS
criteo_day23_raw_data_path: CM_CRITEO_DAY23_RAW_DATA_PATH
use_dataset_from_host: CM_USE_DATASET_FROM_HOST

new_state_keys:
- app_mlperf_inference_*
Expand Down
Loading