Improve examples

[AlexanderDokuchaev]

Changes

• Add pattern based on file extension to gitignore for examples
• Use pathlib style instead os.path
• Fix issue with creating ../nncf/~/.cache/datasets directory
• Replace tqdm to nncf.common.logging.track_progress
• Remove unused function arguments and if conditions
• Clean logs, disable progress bar, remove log of validataion
• Pass list of command arguments to subprocess instead of string
• Remove nncf dependencies from tests/cross_fw/examples
• Add --reuse-venv to use venv from example directory
• Add progress bar to collect_values_for_each_item_using_prepared_model

Tests

https://github.com/openvinotoolkit/nncf/actions/runs/11900403097
nightly/job/test_examples/618
nightly/job/windows/job/test-examples/278/

[MaximProshin]

@AlexanderDokuchaev , @alexsu52 , please run SDL checks before the merge (as you changed some imports and the way how 'path' is used). Also make sure it works on Windows.

[AlexanderDokuchaev]

@MaximProshin
/sdl/job/nightly_trigger/346 - pass, exclude checkmarkx: warning about .github/..
nightly/job/windows/job/test-examples/278/

[KodiaqQ]

Why datasets/** was removed?
In case of the running tests locally, datasets directory may appear and be unintentionally committed.

[AlexanderDokuchaev]

datasets directory created by ultralitics, it's only happens if you have yolo config .config/Ultralytics/settings.json file that was generated some time ago or changes manually.

On current version of yolo, default path for datasets_dir generated as nncf/../datasest if run example from NNCF in case of .config/Ultralytics/settings.json is not exist.
If datasets directory created on your host you need to modify config by yolo settings dataset_dir=new_path or by text editor, or remove config file after it will be generated this new path.

[KodiaqQ]

I do not agree.
I've run the following command from the nncf root (as usual, for testing purposes):
pytest -s tests/cross_fw/examples/test_examples.py -k "post_training_quantization_openvino_yolo8_quantize"
And after the passing, I observe the following directory:

Thus, removing datasets/** from the .gitignore might cause issues.

[KodiaqQ]

From a high perspective, it would be more helpful if you would update examples to create a datasets folder during the run in a corresponding or temporary directory, to exclude incidents in the future. In this case, we would be able to remove this path from .gitignore and safely run tests.

[AlexanderDokuchaev]

Have you updated the ultralytics config file as described above?

[KodiaqQ]

No. I did nothing with the ultralytics package. I've reproduced simple scenario with the following steps:

1. Create clean virtual env;
2. Clone nncf from GitHub;
3. Install nncf & pytest;
4. Run test;

From the contributor's perspective, I would not even know anything about ultralytics managing at all.

[AlexanderDokuchaev]

You have config file ~/.config/Ultralytics/settings.json on your host that was generated when you run any code with ultralitics for the first time and creating or removing any virtual environment does not affect on the config in any way.

Your config collect line like "datasets_dir": "/home/<username>/nncf/datasets",.
This line only generated if run NNCF examples with old version of ultralitics, current default value is like /home/<username>/datasets - GIT_ROOT.parent / "datasets".

There is only one way conditions to get datasets_dir like .../nncf/datasets:

• You did not run any code with ultralitics before
• And for the first time you have run NNCF yolo example with old version of ultralitics

In all other cases datasets_dir will not set as nncf/datasets

My point, If you have config file of any third party package with not default values in your ~/.config directory, that used NNCF folder as storage for anything, it's not responsibility of NNCF, it's incorrect configuration that should be resolved by user.

Moreover, the datasets directory in the NNCF git root folder breaks isort, by changing the import order of datasets module.
So datasets in .gitignore hides this problem by making the existence of this directory expected but it is not.

@alexsu52 as you have added datasets to .gitignore, i think last word is your

[KodiaqQ]

it's not responsibility of NNCF, it's incorrect configuration that should be resolved by user.

If I run NNCF tests from the root and NNCF creates a datasets folder in the root, this is NNCF's responsibility, not the user, since it is not possible to set up the folder for this via tests.
As you said,

datasets directory in the NNCF git root folder breaks isort

So, this is another reason to change the placing of the datasets folder on the NNCF's side, since tests were designed by the NNCF team.

[alexsu52]

I read your discussion and did the following experiment:

• remove ~/.config/Ultralytics/settings.json
• create a clear env using

pip install -r examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/requirements.txt

Ultralytics version:

• run example

python examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/main.py

dataset was downloaded into nncf/datasets

Where did I go wrong?

[KodiaqQ]

@AlexanderDokuchaev, feel free to merge after approval from the other reviewers.

Not blocking merge.