Skip to content

Commit

Permalink
Merge branch 'main' into lint_pylint
Browse files Browse the repository at this point in the history
  • Loading branch information
leouieda committed Feb 17, 2024
2 parents 7705046 + 77c78a2 commit f6104fe
Show file tree
Hide file tree
Showing 19 changed files with 490 additions and 184 deletions.
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
14 changes: 7 additions & 7 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ jobs:
# We pin the commit hash corresponding to v0.5.0, and not pinning the tag
# because we are giving full access through the github.token.
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@148d9a848c6acaf90a3ec30bc5062f646f8a4163
uses: styfle/cancel-workflow-action@0.12.1
with:
access_token: ${{ github.token }}

# Checks-out your repository under $GITHUB_WORKSPACE
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
# Need to fetch more than the last commit so that setuptools-scm can
# create the correct version string. If the number of commits since
Expand Down Expand Up @@ -86,7 +86,7 @@ jobs:
echo "::set-output name=dir::$(pip cache dir)"
- name: Setup caching for pip packages
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('requirements-full.txt') }}
Expand All @@ -113,7 +113,7 @@ jobs:

# Store the docs as a build artifact so we can deploy it later
- name: Upload HTML documentation as an artifact
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: docs-${{ github.sha }}
path: doc/_build/html
Expand All @@ -127,17 +127,17 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4

# Fetch the built docs from the "build" job
- name: Download HTML documentation artifact
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: docs-${{ github.sha }}
path: doc/_build/html

- name: Checkout the gh-pages branch in a separate folder
uses: actions/checkout@5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f
uses: actions/checkout@v4
with:
ref: gh-pages
# Checkout to this folder instead of the current one
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
steps:
# Checks-out your repository under $GITHUB_WORKSPACE
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
# Need to fetch more than the last commit so that setuptools_scm can
# create the correct version string. If the number of commits since
Expand Down Expand Up @@ -79,7 +79,7 @@ jobs:
- name: Upload archives as artifacts
# Only if not a pull request
if: success() && github.event_name != 'pull_request'
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: pypi-${{ github.sha }}
path: dist
Expand All @@ -94,23 +94,23 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
# The GitHub token is preserved by default but this job doesn't need
# to be able to push to GitHub.
persist-credentials: false

# Fetch the built archives from the "build" job
- name: Download built archives artifact
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: pypi-${{ github.sha }}
path: dist

- name: Publish to Test PyPI
# Only publish to TestPyPI when a PR is merged (pushed to main)
if: success() && github.event_name == 'push'
uses: pypa/gh-action-pypi-publish@bce3b74dbf8cc32833ffba9d15f83425c1a736e0
uses: pypa/gh-action-pypi-publish@v1.8.11
with:
user: __token__
password: ${{ secrets.TEST_PYPI_TOKEN}}
Expand All @@ -122,7 +122,7 @@ jobs:
- name: Publish to PyPI
# Only publish to PyPI when a release triggers the build
if: success() && github.event_name == 'release'
uses: pypa/gh-action-pypi-publish@bce3b74dbf8cc32833ffba9d15f83425c1a736e0
uses: pypa/gh-action-pypi-publish@v1.8.11
with:
user: __token__
password: ${{ secrets.PYPI_TOKEN}}
4 changes: 2 additions & 2 deletions .github/workflows/style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
persist-credentials: false

Expand All @@ -42,7 +42,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
persist-credentials: false

Expand Down
26 changes: 14 additions & 12 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,6 @@ on:
release:
types:
- published
schedule:
# Run every Monday at 12:00 UTC
# * is a special character in YAML so you have to quote this string
- cron: "00 12 * * 1"

# Use bash by default in all jobs
defaults:
Expand All @@ -43,13 +39,17 @@ jobs:
- ubuntu
- macos
- windows
python:
- "3.7"
- "3.11"
dependencies:
- oldest
- latest
- optional
- oldest
include:
- dependencies: oldest
python: "3.7"
- dependencies: latest
python: "3.11"
- dependencies: optional
python: "3.11"
env:
REQUIREMENTS: env/requirements-build.txt env/requirements-test.txt
# Used to tag codecov submissions
Expand All @@ -62,13 +62,13 @@ jobs:
# We pin the commit hash corresponding to v0.5.0, and not pinning the tag
# because we are giving full access through the github.token.
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@148d9a848c6acaf90a3ec30bc5062f646f8a4163
uses: styfle/cancel-workflow-action@0.12.1
with:
access_token: ${{ github.token }}

# Checks-out your repository under $GITHUB_WORKSPACE
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
# Need to fetch more than the last commit so that setuptools-scm can
# create the correct version string. If the number of commits since
Expand Down Expand Up @@ -119,7 +119,7 @@ jobs:
echo "::set-output name=dir::$(pip cache dir)"
- name: Setup caching for pip packages
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('requirements-full.txt') }}
Expand Down Expand Up @@ -148,11 +148,13 @@ jobs:
run: coverage xml

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
files: ./coverage.xml
env_vars: OS,PYTHON,DEPENDENCIES
# Don't mark the job as failed if the upload fails for some reason.
# It does sometimes but shouldn't be the reason for running
# everything again unless something else is broken.
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
75 changes: 32 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,35 +21,41 @@ Part of the <a href="https://www.fatiando.org"><strong>Fatiando a Terra</strong>

## About

*Does your Python package include sample datasets?
Are you shipping them with the code?
Are they getting too big?*
> Just want to download a file without messing with `requests` and `urllib`?
> Trying to add sample datasets to your Python package?
> **Pooch is here to help!**
**Pooch** is here to help! It will manage a data *registry* by downloading your
data files from a server only when needed and storing them locally in a data
*cache* (a folder on your computer).

Here are Pooch's main features:
*Pooch* is a **Python library** that can manage data by **downloading files**
from a server (only when needed) and storing them locally in a data **cache**
(a folder on your computer).

* Pure Python and minimal dependencies.
* Download a file only if necessary (it's not in the data cache or needs to be
updated).
* Verify download integrity through SHA256 hashes (also used to check if a file
needs to be updated).
* Designed to be extended: plug in custom download (FTP, scp, etc) and
post-processing (unzip, decompress, rename) functions.
* Includes utilities to unzip/decompress the data upon download to save loading
time.
* Can handle basic HTTP authentication (for servers that require a login) and
printing download progress bars.
* Easily set up an environment variable to overwrite the data cache location.

*Are you a scientist or researcher? Pooch can help you too!*

* Automatically download your data files so you don't have to keep them in your
GitHub repository.
* Make sure everyone running the code has the same version of the data files
(enforced through the SHA256 hashes).
* Download files over HTTP, FTP, and from data repositories like Zenodo and figshare.
* Built-in post-processors to unzip/decompress the data after download.
* Designed to be extended: create custom downloaders and post-processors.

Are you a **scientist** or researcher? Pooch can help you too!

* Host your data on a repository and download using the DOI.
* Automatically download data using code instead of telling colleagues to do it themselves.
* Make sure everyone running the code has the same version of the data files.

## Projects using Pooch

[SciPy](https://github.com/scipy/scipy),
[scikit-image](https://github.com/scikit-image/scikit-image),
[Ensaio](https://github.com/fatiando/ensaio),
[MetPy](https://github.com/Unidata/MetPy),
[napari](https://github.com/napari/napari),
[icepack](https://github.com/icepack/icepack),
[histolab](https://github.com/histolab/histolab),
[seaborn-image](https://github.com/SarthakJariwala/seaborn-image),
[Open AR-Sandbox](https://github.com/cgre-aachen/open_AR_Sandbox),
[climlab](https://github.com/climlab/climlab),
[mne-python](https://github.com/mne-tools/mne-python),
[GemGIS](https://github.com/cgre-aachen/gemgis)

> If you're using Pooch, **send us a pull request** adding your project to the list.
## Example

Expand Down Expand Up @@ -136,23 +142,6 @@ def fetch_gravity_data():
return data
```

## Projects using Pooch

* [SciPy](https://github.com/scipy/scipy)
* [scikit-image](https://github.com/scikit-image/scikit-image)
* [MetPy](https://github.com/Unidata/MetPy)
* [icepack](https://github.com/icepack/icepack)
* [histolab](https://github.com/histolab/histolab)
* [seaborn-image](https://github.com/SarthakJariwala/seaborn-image)
* [Ensaio](https://github.com/fatiando/ensaio)
* [Open AR-Sandbox](https://github.com/cgre-aachen/open_AR_Sandbox)
* [climlab](https://github.com/climlab/climlab)
* [napari](https://github.com/napari/napari)
* [mne-python](https://github.com/mne-tools/mne-python)
* [GemGIS](https://github.com/cgre-aachen/gemgis)

*If you're using Pooch, send us a pull request adding your project to the list.*

## Getting involved

🗨️ **Contact us:**
Expand Down
Binary file modified doc/_static/favicon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion doc/_templates/layout.html
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@


{% block htmltitle %}
{% if title == '' or title == 'Home' %}
{% if title == '' or 'no title' in title or title == 'Home' %}
<title>{{ docstitle|striptags|e }}</title>
{% else %}
<title>{{ title|striptags|e }} | {{ docstitle|striptags|e }}</title>
Expand Down
53 changes: 27 additions & 26 deletions doc/about.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,42 +6,43 @@ Why use Pooch?
Use cases
---------

.. tab-set::

.. tabbed:: Just download a file
.. tab-item:: Just download a file

**Who**: Scientists/researchers/developers looking to simply download a
file.
**Who**: Scientists/researchers/developers looking to simply download a
file.

Pooch makes it easy to download a file (one function call).
On top of that, it also comes with some bonus features:
Pooch makes it easy to download a file (one function call).
On top of that, it also comes with some bonus features:

* Download and cache your data files locally (so it's only downloaded
once).
* Make sure everyone running the code has the same version of the data
files by verifying cryptographic hashes.
* Multiple download protocols HTTP/FTP/SFTP and basic authentication.
* Download from Digital Object Identifiers (DOIs) issued by repositories
like figshare and Zenodo.
* Built-in utilities to unzip/decompress files upon download
* Download and cache your data files locally (so it's only downloaded
once).
* Make sure everyone running the code has the same version of the data
files by verifying cryptographic hashes.
* Multiple download protocols HTTP/FTP/SFTP and basic authentication.
* Download from Digital Object Identifiers (DOIs) issued by repositories
like figshare and Zenodo.
* Built-in utilities to unzip/decompress files upon download

**Start here:** :ref:`retrieve`
**Start here:** :ref:`retrieve`

.. tabbed:: Use by another library
.. tab-item:: Manage sample data for a Python program

**Who**: Package developers wanting to include sample data for use in
tutorials and tests.
**Who**: Package developers wanting to include sample data for use in
tutorials and tests.

Pooch was designed for this! It offers:
Pooch was designed for this! It offers:

* Pure Python and :ref:`minimal dependencies <dependencies>`.
* Download a file only if necessary.
* Verification of download integrity through cryptographic hashes.
* Extensible design: plug in custom download and post-processing functions.
* Built-in utilities to unzip/decompress files upon download
* Multiple download protocols HTTP/FTP/SFTP and basic authentication.
* User control of data cache location through environment variables.
* Pure Python and :ref:`minimal dependencies <dependencies>`.
* Download a file only if necessary.
* Verification of download integrity through cryptographic hashes.
* Extensible design: plug in custom download and post-processing functions.
* Built-in utilities to unzip/decompress files upon download
* Multiple download protocols HTTP/FTP/SFTP and basic authentication.
* User control of data cache location through environment variables.

**Start here:** :ref:`intermediate`
**Start here:** :ref:`intermediate`

History
-------
Expand Down
Loading

0 comments on commit f6104fe

Please sign in to comment.