Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch execution listing experiment folders wrongly #952

Open
brunofavs opened this issue May 10, 2024 · 2 comments
Open

Batch execution listing experiment folders wrongly #952

brunofavs opened this issue May 10, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@brunofavs
Copy link
Collaborator

This line :

# get all runs in this experiment
files_and_folders = glob.glob(args['results_folder'] + '/' + experiment + '*')
files_and_folders.sort()

It's not doing what it's supposed to be doing.



['results/nig_0.1-ntfv_0.25_run001_fold001',
'results/nig_0.1-ntfv_0.25_run001_fold002',
'results/nig_0.1-ntfv_0.25_run001_fold003',
'results/nig_0.1-ntfv_0.25_run001_fold004',
'results/nig_0.1-ntfv_0.25_run001_fold005',
'results/nig_0.1-ntfv_0.25_run002_fold001',
'results/nig_0.1-ntfv_0.25_run002_fold002',
'results/nig_0.1-ntfv_0.25_run002_fold003',
'results/nig_0.1-ntfv_0.25_run002_fold004',
'results/nig_0.1-ntfv_0.25_run002_fold005',
'results/nig_0.1-ntfv_0.25_run003_fold001',
'results/nig_0.1-ntfv_0.25_run003_fold002',
'results/nig_0.1-ntfv_0.25_run003_fold003',
'results/nig_0.1-ntfv_0.25_run003_fold004',
'results/nig_0.1-ntfv_0.25_run003_fold005']
results/nig_0.1-ntfv_0.25_run001_fold001


['results/nig_0.15-ntfv_0.15_run001_fold001',
'results/nig_0.15-ntfv_0.15_run001_fold002',
'results/nig_0.15-ntfv_0.15_run001_fold003',
'results/nig_0.15-ntfv_0.15_run001_fold004',
'results/nig_0.15-ntfv_0.15_run001_fold005',
'results/nig_0.15-ntfv_0.15_run002_fold001',
'results/nig_0.15-ntfv_0.15_run002_fold002',
'results/nig_0.15-ntfv_0.15_run002_fold003',
'results/nig_0.15-ntfv_0.15_run002_fold004',
'results/nig_0.15-ntfv_0.15_run002_fold005',
'results/nig_0.15-ntfv_0.15_run003_fold001',
'results/nig_0.15-ntfv_0.15_run003_fold002',
'results/nig_0.15-ntfv_0.15_run003_fold003',
'results/nig_0.15-ntfv_0.15_run003_fold004',
'results/nig_0.15-ntfv_0.15_run003_fold005',
'results/nig_0.15-ntfv_0.1_run001_fold001',
'results/nig_0.15-ntfv_0.1_run001_fold002',
'results/nig_0.15-ntfv_0.1_run001_fold003',
'results/nig_0.15-ntfv_0.1_run001_fold004',
'results/nig_0.15-ntfv_0.1_run001_fold005',
'results/nig_0.15-ntfv_0.1_run002_fold001',
'results/nig_0.15-ntfv_0.1_run002_fold002',
'results/nig_0.15-ntfv_0.1_run002_fold003',
'results/nig_0.15-ntfv_0.1_run002_fold004',
'results/nig_0.15-ntfv_0.1_run002_fold005',
'results/nig_0.15-ntfv_0.1_run003_fold001',
'results/nig_0.15-ntfv_0.1_run003_fold002',
'results/nig_0.15-ntfv_0.1_run003_fold003',
'results/nig_0.15-ntfv_0.1_run003_fold004',
'results/nig_0.15-ntfv_0.1_run003_fold005']

This is the print of the said line. When grepping the nig 0.15 ntfv 0.15 experiment it grabs wrong things with it as well.

@brunofavs brunofavs added the bug Something isn't working label May 10, 2024
@brunofavs brunofavs self-assigned this May 10, 2024
brunofavs added a commit that referenced this issue May 10, 2024
Just commiting a hunk of a bigger change in this file. Working on #951 atm
@brunofavs
Copy link
Collaborator Author

brunofavs commented May 10, 2024

Seems to be "fixed".

What was happening is that
glob.glob(args['results_folder'] + '/' + experiment + '*') with the nig_0.15-ntfv_0.1 experiment was also grepping the nig_0.15-ntfv_0.15.

Adding a "_" should fixed all cases since AS LONG AS we make sure to not use underscores in the experiment names.

A experience named nig_0.15-ntfv_0.1_banana would break in the same way it was breaking now.

I'm not the biggest fan of this solution because it's prone to breaking sneakily. There's not a lot we can do. One will always be a substring of the other.

What we can do is rewrite any underscores in the naming to hyphens in the batch execution. Like a safeguard, leaving underscore as a escape character. That or substitute the underscore in _fold_run for a very unusual escape character, like «fold«run or something, albeit I agree it would look kinda funky.

What is your opinion on this @miguelriemoliveira ?

@brunofavs
Copy link
Collaborator Author

That or a way more complicated filtering mechanism, considering now the experience settings are also on a yaml file for each folder, which is more bulletproof than having the settings encoded in the name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

1 participant