Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add checks to ensure required files/directories exist #36

Open
skyfenton opened this issue Nov 1, 2024 · 2 comments
Open

Add checks to ensure required files/directories exist #36

skyfenton opened this issue Nov 1, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@skyfenton
Copy link
Collaborator

skyfenton commented Nov 1, 2024

Add checks to create directories and files, and/or warn the user they don't have the required files (ex. netflix dataset downloaded, eventually recommendation model checkpoint file, etc.). Currently, we just run the program until it fails (ex. whole processing script runs and then no "/out" directory exists). I'm sure this is a obvious issue, but I thought I'd document it so we don't forget + can discuss file structure.

I think this could be added to #28, but we might want to have an extendable way to check for files we require isolated to a designated file.

@skyfenton skyfenton added the enhancement New feature or request label Nov 1, 2024
@audiodude
Copy link
Collaborator

Yes, we should create the required directories here: https://github.com/noisebridge/MediaBridge/blob/main/mediabridge/data_processing/wiki_to_netflix.py#L14-L15

os.makedirs(data_dir, exist_ok=True), same thing for out_dir.

@audiodude
Copy link
Collaborator

As far as not having the Netflix data, we could definitely add some code that downloads it and decompresses it to the right location. Or we can just wrap the part that tries to read the movie titles in a try...except and print a more useful message ("It seems you don't have the Netflix dataset download, please get it from https://....").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants