Merge branch 'mlperf-inference' into mixtral+gha+selfhosted

README.md

@@ -13,125 +13,10 @@
 [![Test QAIC Software kit Compilation](https://github.com/mlcommons/cm4mlops/actions/workflows/test-qaic-software-kit.yml/badge.svg)](https://github.com/mlcommons/cm4mlops/actions/workflows/test-qaic-software-kit.yml)
 
 
-This repository contains reusable and cross-platform automation recipes to run DevOps, MLOps, and MLPerf 
-via a simple and human-readable [Collective Mind interface (CM)](https://github.com/mlcommons/ck) 
-while adapting to different operating systems, software and hardware.
+The `mlperf-branch` of the **cm4mlops** repository contains updated CM scripts specifically for MLPerf Inference, including support for Automotive. Please note that the general CM scripts in this branch may not be compatible with other projects. For more information on using CM for MLPerf Inference, visit the [MLPerf Inference Documentation site](https://docs.mlcommons.org/inference/).
 
-All СM scripts have a simple Python API, extensible JSON/YAML meta description
-and unified input/output to make them reusable in different projects either individually 
-or by chaining them together into portable automation workflows, applications 
-and web services adaptable to continuously changing models, data sets, software and hardware.
+[![Streamline your MLPerf results using CM Framework](https://img.youtube.com/vi/eI1Hoecc3ho/0.jpg)](https://youtu.be/eI1Hoecc3ho)
 
-We develop and test [CM scripts](script) as a community effort to support the following projects:
-* [CM for MLPerf](https://docs.mlcommons.org/inference): modularize and automate MLPerf benchmarks 
-  * [Modular C++ harness for MLPerf loadgen](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-mlcommons-cpp)
-  * [Modular Python harness for MLPerf loadgen](https://github.com/mlcommons/cm4mlops/tree/main/script/app-loadgen-generic-python)
-* [CM for research and education](https://cTuning.org/ae): provide a common interface to automate and reproduce results from research papers 
-  and MLPerf benchmarks;
-* [CM for ABTF](https://github.com/mlcommons/cm4abtf): provide a unified CM interface to run automotive benchmarks;
-* [CM for optimization](https://access.cknowledge.org/playground/?action=challenges): co-design efficient and cost-effective 
-  software and hardware for AI, ML and other emerging workloads via open challenges. 
-
-You can read this [ArXiv paper](https://arxiv.org/abs/2406.16791) to learn more about the CM motivation and long-term vision.
-
-Please provide your feedback or submit your issues [here](https://github.com/mlcommons/cm4mlops/issues).
-
-
-## Catalog
-
-Online catalog: [cKnowledge](https://access.cknowledge.org/playground/?action=scripts), [MLCommons](https://docs.mlcommons.org/cm4mlops/scripts).
-
-## Citation
-
-Please use this [BibTeX file](https://github.com/mlcommons/ck/blob/master/citation.bib) to cite this project.
-
-## A few demos
-
-### Install CM and virtual env
-
-Install the [MLCommons CM automation language](https://access.cknowledge.org/playground/?action=install).
-
-### Pull this repository
-
-```bash
-cm pull repo mlcommons@cm4mlops --branch=dev
-```
-
-### Run image classification using CM
-
-```bash
-
-cm run script "python app image-classification onnx _cpu" --help
-
-cm run script "download file _wget" --url=https://cKnowledge.org/ai/data/computer_mouse.jpg --verify=no --env.CM_DOWNLOAD_CHECKSUM=45ae5c940233892c2f860efdf0b66e7e
-cm run script "python app image-classification onnx _cpu" --input=computer_mouse.jpg
-
-cmr "python app image-classification onnx _cpu" --input=computer_mouse.jpg
-cmr --tags=python,app,image-classification,onnx,_cpu --input=computer_mouse.jpg
-cmr 3d5e908e472b417e --input=computer_mouse.jpg
-
-cm docker script "python app image-classification onnx _cpu" --input=computer_mouse.jpg
-
-cm gui script "python app image-classification onnx _cpu"
-```
-
-### Re-run experiments from the ACM/IEEE MICRO'23 paper
-
-Check this [script/reproduce-ieee-acm-micro2023-paper-96](README.md).
-
-### Run MLPerf ResNet CPU inference benchmark via CM
-
-```bash
-cm run script --tags=run-mlperf,inference,_performance-only,_short  \
-   --division=open \
-   --category=edge \
-   --device=cpu \
-   --model=resnet50 \
-   --precision=float32 \
-   --implementation=mlcommons-python \
-   --backend=onnxruntime \
-   --scenario=Offline \
-   --execution_mode=test \
-   --power=no \
-   --adr.python.version_min=3.8 \
-   --clean \
-   --compliance=no \
-   --quiet \
-   --time
-```
-
-### Run MLPerf BERT CUDA inference benchmark v4.1 via CM
-
-```bash
-cmr "run-mlperf inference _find-performance _full _r4.1" \
-   --model=bert-99 \
-   --implementation=nvidia \
-   --framework=tensorrt \
-   --category=datacenter \
-   --scenario=Offline \
-   --execution_mode=test \
-   --device=cuda  \
-   --docker \
-   --docker_cm_repo=mlcommons@cm4mlops \
-   --docker_cm_repo_flags="--branch=mlperf-inference" \
-   --test_query_count=100 \
-   --quiet
-```
-
-### Run MLPerf SDXL reference inference benchmark v4.1 via CM
-
-```bash
-cm run script \
-	--tags=run-mlperf,inference,_r4.1 \
-	--model=sdxl \
-	--implementation=reference \
-	--framework=pytorch \
-	--category=datacenter \
-	--scenario=Offline \
-	--execution_mode=valid \
-	--device=cuda \
-	--quiet
-```
 
 
 ## License

script/push-mlperf-inference-results-to-github/run.bat

@@ -17,7 +17,7 @@ git add *
 
 REM Check if the CM_MLPERF_INFERENCE_SUBMISSION_DIR variable is set
 if defined CM_MLPERF_INFERENCE_SUBMISSION_DIR (
-    robocopy "%CM_MLPERF_INFERENCE_SUBMISSION_DIR%" "%CM_GIT_REPO_CHECKOUT_PATH%" /MIR
+    robocopy "%CM_MLPERF_INFERENCE_SUBMISSION_DIR%" "%CM_GIT_REPO_CHECKOUT_PATH%" /MIR /A+H
     git add *
 )