Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load and save from/to disk no longer accept pathlib.Path #6829

Open
albertvillanova opened this issue Apr 23, 2024 · 0 comments · May be fixed by #6828
Open

Load and save from/to disk no longer accept pathlib.Path #6829

albertvillanova opened this issue Apr 23, 2024 · 0 comments · May be fixed by #6828
Labels
bug Something isn't working

Comments

@albertvillanova
Copy link
Member

Reported by @vttrifonov at #6704 (comment):

This change is breaking in

datasets/src/datasets/arrow_dataset.py

Line 1515 in f96e74d

fs, _ = url_to_fs(dataset_path, **(storage_options or {}))

when the input is pathlib.Path. The issue is that url_to_fs expects a str and cannot deal with Path. get_fs_token_paths converts to str so it is not a problem

This change was introduced in:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support PathLike input in save_to_disk / load_from_disk huggingface/datasets
1 participant
@albertvillanova