Model tf_2dunet - Plan initialisation fails expecting /raid/datasets/MICCAI_BraTS_2019_Data_Training/HGG/0

[noopurintel]

Describe the bug
While trying the Quick Start Guide for model tf_2dunet, the plan initialisation step is failing.

Last few lines from the error message:

File "/home/azureuser/openfl/tests/openfl_e2e/my_workspace/src/tfbrats_inmemory.py", line 29, in __init__
    X_train, y_train, X_valid, y_valid = load_from_nifti(parent_dir=data_path,
  File "/home/azureuser/openfl/tests/openfl_e2e/my_workspace/src/brats_utils.py", line 94, in load_from_nifti
    subdirs = os.listdir(path)
FileNotFoundError: [Errno 2] No such file or directory: '/raid/datasets/MICCAI_BraTS_2019_Data_Training/HGG/0'

To Reproduce
Steps to reproduce the behavior:

1. Follow the steps mentioned in Quick Start replacing model torch_cnn_mnist with tf_2dunet
2. Create workspace, certify it.
3. Generate CSR request for aggregator with CA signing it.
4. Initialise the plan - fx plan initialize

At this step the error is thrown.

Expected behavior
There should be no error during plan initialisation.

Screenshots

Machine

• Ubuntu 22.04

Additional

There is this README.md which mentions dataset structure for MICCAI_BraTS_2019_Data_Training.
But how to download it exactly? Is this mentioned anywhere?

For practice purpose, I found this link having dataset - https://www.kaggle.com/datasets/aryashah2k/brain-tumor-segmentation-brats-2019 but it contains too many subfolders as opposed to expected 0 and 1.

[kta-intel]

fx plan initialize is currently taking the first entry from data.yaml. You either need to directly overwrite this to point at your dataset, or you can invoke the --input_shape flag if you know the expected data shape

To gain access to the data, originally you needed to send an access request to the MICCAI BraTS challenge, but that Kaggle link actually looks like the proper data. If so, the README.md includes steps to shard the data

[theakshaypant]

If one is able to run an experiment after the fix, should also consider closing #366 #398