Skip to content

Commit

Permalink
Add FileDownloader class with file protocol to allow repository to be…
Browse files Browse the repository at this point in the history
… e.g. on a Network share.

Path needs to start with file://.
  • Loading branch information
gyger committed Jul 2, 2024
1 parent 416f7b6 commit 75e9edb
Showing 1 changed file with 62 additions and 0 deletions.
62 changes: 62 additions & 0 deletions pooch/downloaders.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ def choose_downloader(url, progressbar=False):
"http": HTTPDownloader,
"sftp": SFTPDownloader,
"doi": DOIDownloader,
"file": FileDownloader,
}

parsed_url = parse_url(url)
Expand Down Expand Up @@ -502,6 +503,67 @@ def callback(current, total):
if sftp is not None:
sftp.close()

class FileDownloader: # pylint: disable=too-few-public-methods
"""
Download manager for fetching files over a file system mounted in the operating system.
When called, downloads the given file path into the specified local file.
Uses :mod:`shutil` to copy files from paths.
Note: Does not support a progressbar.
"""

def __init__(self, progressbar=False, chunk_size=1024, **kwargs):
self.kwargs = kwargs
self.progressbar = progressbar
self.chunk_size = chunk_size

Check warning on line 519 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L517-L519

Added lines #L517 - L519 were not covered by tests

def __call__(
self, url, output_file, pooch, check_only=False
): # pylint: disable=R0914
"""
Download the given file in the filesystem to the given output file.
Uses :func:`shutil.copyfile` or :func:`shutil.copyfileobj`.
Parameters
----------
url : str
The url path to the file you want to download.
output_file : str or file-like object
Path (and file name) to which the file will be downloaded.
pooch : :class:`~pooch.Pooch`
The instance of :class:`~pooch.Pooch` that is calling this method.
check_only : bool
If True, will only check if a file exists in the directory and
**without downloading the file**. Will return ``True`` if the file
exists and ``False`` otherwise.
Returns
-------
availability : bool or None
If ``check_only==True``, returns a boolean indicating if the file
is available on the server. Otherwise, returns ``None``.
"""
import pathlib # pylint: disable=C0415

Check warning on line 549 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L549

Added line #L549 was not covered by tests

parsed_url = parse_url(url)
source_path = pathlib.Path(parsed_url['netloc'] + parsed_url['path'])

Check warning on line 552 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L551-L552

Added lines #L551 - L552 were not covered by tests

if check_only:
return source_path.exists()

Check warning on line 555 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L554-L555

Added lines #L554 - L555 were not covered by tests

import shutil # pylint: disable=C0415

Check warning on line 557 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L557

Added line #L557 was not covered by tests

ispath = not hasattr(output_file, "write")
if ispath:
shutil.copyfile(source_path, output_file)

Check warning on line 561 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L559-L561

Added lines #L559 - L561 were not covered by tests
else:
with source_path.open('rb') as fsrc:
shutil.copyfileobj(fsrc, output_file)

Check warning on line 564 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L563-L564

Added lines #L563 - L564 were not covered by tests

return None

Check warning on line 566 in pooch/downloaders.py

View check run for this annotation

Codecov / codecov/patch

pooch/downloaders.py#L566

Added line #L566 was not covered by tests

class DOIDownloader: # pylint: disable=too-few-public-methods
"""
Expand Down

0 comments on commit 75e9edb

Please sign in to comment.