Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs

[github-actions]

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link -

beam/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy

Line 151 in 42d0a6e

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7

timestamp: Sun Nov 19 06:36:43 2023, metric_value: 209499.95
timestamp: Sat Nov 18 06:29:46 2023, metric_value: 212258.56
timestamp: Fri Nov 17 06:31:43 2023, metric_value: 209102.95 <---- Anomaly
timestamp: Wed Nov 15 06:30:16 2023, metric_value: 168926.49
timestamp: Tue Nov 14 06:26:12 2023, metric_value: 165426.66
timestamp: Mon Nov 13 06:26:33 2023, metric_value: 173417.09
timestamp: Sun Nov 12 06:26:54 2023, metric_value: 162621.34
timestamp: Sat Nov 11 06:26:23 2023, metric_value: 168426.61
timestamp: Fri Nov 10 06:26:40 2023, metric_value: 168759.09
timestamp: Thu Nov  9 06:29:53 2023, metric_value: 171794.31
timestamp: Tue Nov  7 07:13:42 2023, metric_value: 167979.26
timestamp: Tue Nov  7 06:28:45 2023, metric_value: 172938.72
timestamp: Mon Nov  6 06:35:23 2023, metric_value: 170587.09

[AnandInguva]

Similar to #29491.

cc: @tvalentyn current python interrupts if you could take a look.

[github-actions]

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link -

beam/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy

Line 151 in 42d0a6e

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7

timestamp: Sat Dec 30 06:46:27 2023, metric_value: 202143.13
timestamp: Fri Dec 29 06:45:37 2023, metric_value: 210222.32
timestamp: Thu Dec 28 00:16:10 2023, metric_value: 207040.46
timestamp: Wed Dec 27 06:44:56 2023, metric_value: 217219.49
timestamp: Tue Dec 26 06:44:21 2023, metric_value: 203994.03
timestamp: Sun Dec 24 07:04:48 2023, metric_value: 205072.90
timestamp: Sat Dec 23 06:58:13 2023, metric_value: 205543.48
timestamp: Fri Dec 22 06:52:50 2023, metric_value: 209199.66 <---- Anomaly
timestamp: Thu Dec 21 06:52:04 2023, metric_value: 221007.77
timestamp: Wed Dec 20 07:04:02 2023, metric_value: 223566.49
timestamp: Tue Dec 19 06:54:26 2023, metric_value: 215618.13
timestamp: Mon Dec 18 16:43:04 2023, metric_value: 231189.45
timestamp: Fri Dec 15 06:54:00 2023, metric_value: 217229.54
timestamp: Thu Dec 14 06:45:17 2023, metric_value: 207335.30
timestamp: Wed Dec 13 06:55:32 2023, metric_value: 213023.18
timestamp: Tue Dec 12 06:40:05 2023, metric_value: 213582.44
timestamp: Mon Dec 11 06:37:05 2023, metric_value: 218185.46
timestamp: Sun Dec 10 06:32:59 2023, metric_value: 209092.40

[github-actions]

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link -

beam/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy

Line 151 in 42d0a6e

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7

timestamp: Mon Feb  5 06:48:47 2024, metric_value: 219060.07
timestamp: Sat Feb  3 06:45:12 2024, metric_value: 225485.20
timestamp: Thu Feb  1 06:48:04 2024, metric_value: 221938.53 <---- Anomaly
timestamp: Tue Jan 30 06:47:30 2024, metric_value: 209624.08
timestamp: Mon Jan 29 06:45:18 2024, metric_value: 209521.78
timestamp: Sun Jan 28 06:39:42 2024, metric_value: 200487.08
timestamp: Sat Jan 27 06:43:41 2024, metric_value: 222538.60
timestamp: Fri Jan 26 06:45:21 2024, metric_value: 210453.72
timestamp: Thu Jan 25 06:44:42 2024, metric_value: 208565.78
timestamp: Wed Jan 24 06:55:31 2024, metric_value: 211543.25
timestamp: Mon Jan 22 06:44:51 2024, metric_value: 209070.38
timestamp: Sun Jan 21 06:45:18 2024, metric_value: 201586.58
timestamp: Sat Jan 20 06:38:09 2024, metric_value: 196935.87

[tvalentyn]

I think this benchmark readings have too much variance, doesn't seem to be an anomaly.