Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI testing of unittests and linting #16

Merged
merged 24 commits into from
Jul 27, 2022
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
970e306
Add pre-commit hooks
andrewphilipsmith Jul 25, 2022
963495e
Apply import sorting (isort)
andrewphilipsmith Jul 25, 2022
a599d1e
Enforce PEP8
andrewphilipsmith Jul 25, 2022
40f9192
Enforce `black` formatting
andrewphilipsmith Jul 25, 2022
55985b4
Create an GH Action which runs pytest
andrewphilipsmith Jul 25, 2022
2c7af72
Update test skip conditionals for GH Actions rather than Travis.
andrewphilipsmith Jul 26, 2022
00cc7f1
Port conda test env from Travis to GH Actions
andrewphilipsmith Jul 26, 2022
ec9104f
Tweaked conda env
andrewphilipsmith Jul 26, 2022
33543c8
Fixed syntax error in ci-tests.yml
andrewphilipsmith Jul 26, 2022
56f771c
Try python version as string
andrewphilipsmith Jul 26, 2022
fb61ade
Test separation of conda env / shell between steps
andrewphilipsmith Jul 26, 2022
91a6dba
Set default shell args to pass conda activate env between steps
andrewphilipsmith Jul 26, 2022
c2dae8e
Attempt to run tests with a explicit environment.yml
andrewphilipsmith Jul 26, 2022
d463d14
Fix typo in ci-test.yml
andrewphilipsmith Jul 26, 2022
66ea97e
Limit CI test to just the most problematic test during debugging
andrewphilipsmith Jul 26, 2022
88da202
Don't patch environment.yml
andrewphilipsmith Jul 26, 2022
a0fd4fd
Using minimal environment.yml
andrewphilipsmith Jul 26, 2022
74aca25
Skip test affected by bug #19
andrewphilipsmith Jul 26, 2022
f418db7
Renamed .travis.yml so that it is for reference only.
andrewphilipsmith Jul 26, 2022
77bbdff
Only disable `test_combine` on CI, but not locally
andrewphilipsmith Jul 26, 2022
93cbf73
Update from main
andrewphilipsmith Jul 27, 2022
1412d84
Fix linting errors
andrewphilipsmith Jul 27, 2022
a5b0fb1
Add workaround for issue #20
andrewphilipsmith Jul 27, 2022
c0a4c87
Fix missing import
andrewphilipsmith Jul 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[flake8]
max-line-length = 120
ignore = E203
44 changes: 44 additions & 0 deletions .github/workflows/ci-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Unit tests

on:
pull_request:
branches:
- main
push:
# Run on main because caches are inherited from parent branches

jobs:
build:

runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.7"]

# Set default shell to allow conda activate env to be passed from one step to the next
defaults:
run:
shell: bash -l {0}

steps:
- uses: actions/checkout@v2
- name: Setup Miniconda with Python ${{ matrix.python-version }}
uses: conda-incubator/[email protected]
with:
python-version: ${{ matrix.python-version }}
environment-file: environment.yml
clean-patched-environment-file: false
- name: Check Conda configuration
run: |
conda info -a
python --version
- name: Install repo using pip
run: |
python -m pip install .
- name: Test with pytest
run: |
# Limit to just the most problematic test during debugging
pytest --durations=0
41 changes: 41 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
# Adapted from https://github.com/alan-turing-institute/Python-quality-tools
name: pre-commit action
on:
pull_request:
branches:
- main
push:
# Run on main because caches are inherited from parent branches
env:
# This should be the default but we'll be explicit
PRE_COMMIT_HOME: ~/.caches/pre-commit
jobs:
pre-commit_job:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: "3.7"
- name: Install Pre-Commit
shell: bash
run: |
python -m pip install pre-commit
- name: Cache Pre-Commit Hooks
id: cache
uses: actions/cache@v3
with:
path: ${{ env.PRE_COMMIT_HOME }}
key: ${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Install Pre-Commit Hooks
shell: bash
if: steps.cache.outputs.cache-hit != 'true'
run: |
pre-commit install-hooks
- name: Run Pre-Commit Hooks
shell: bash
run: |
pre-commit run --all-files
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -113,4 +113,10 @@ pyveg/azure_config.py
#docs things
docs/build
docs/source/_static
docs/source/_templates
docs/source/_templates

# Test data
pyveg/testdata/test_ne_england/

# Virtual Environments
env/
38 changes: 38 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks

# Ignore RVeg and Notebooks for now
exclude: (^rveg|^notebooks|^binder)
default_language_version:
# force all unspecified python hooks to run python3
python: python3.7
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-xml
- id: check-added-large-files
- repo: https://github.com/psf/black
rev: 22.3.0
hooks:
- id: black
exclude: (^pyveg/configs|^pyveg/zenodo_config_template.py|^pyveg/azure_config_template.py)
- repo: https://github.com/pre-commit/mirrors-autopep8
rev: 'v1.6.0' # Use the sha / tag you want to point at
hooks:
- id: autopep8
- repo: https://github.com/pycqa/isort
rev: 5.10.1
hooks:
- id: isort
args: ["--profile", "black", "--filter-files"]

# Ignore type checking for now, as this would require non-trival change to the code
# in order to pass `mypy` tests
# - repo: https://github.com/pre-commit/mirrors-mypy
# rev: 'v0.960' # Use the sha / tag you want to point at
# hooks:
# - id: mypy
21 changes: 12 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,17 @@ Don't let trying to be perfect get in the way of being good - exciting ideas are

## Table of contents

- [Where to start: issues](#where-to-start-issues)
- [Making a change with a pull request](#making-a-change-with-a-pull-request)
- [1. Comment on an existing issue or open a new issue referencing your addition](#1-comment-on-an-existing-issue-or-open-a-new-issue-referencing-your-addition)
- [2. Create a new branch (if you have *write* access to the repository) or fork the repository to your profile (if you don't currently have _write_ access)](#2-create-a-new-branch-or-fork-the-repository-to-your-profile)
- [3. Make the changes you've discussed](#3-make-the-changes-youve-discussed)
- [4. Submit a pull request](#4-submit-a-pull-request)
- [Style guide](#style-guide)
- [Contributing to monitoring-ecosystem-resilience (the repo!)](#contributing-to-monitoring-ecosystem-resilience-the-repo)
- [Table of contents](#table-of-contents)
- [Where to start: issues](#where-to-start-issues)
- [Making a change with a pull request](#making-a-change-with-a-pull-request)
- [1. Comment on an existing issue or open a new issue referencing your addition](#1-comment-on-an-existing-issue-or-open-a-new-issue-referencing-your-addition)
- [2. Create a new branch to your profile](#2-create-a-new-branch-or-forkgithub-fork-the-monitoring-ecosystem-resilience-repositorymonitoring-ecosystem-resilience-repo-to-your-profile)
- [2a) Create a branch](#2a-create-a-branch)
- [2b. Fork the repository](#2b-fork-the-repository)
- [3. Make the changes you've discussed](#3-make-the-changes-youve-discussed)
- [4. Submit a pull request](#4-submit-a-pull-request)
- [Style Guide](#style-guide)

## Where to start: issues

Expand Down Expand Up @@ -138,8 +142,7 @@ Simply make your edits on their files and open a pull request to their branch!

What happens if the continuous integration (CI) fails (for example, if the pull request notifies you that "Some checks were not successful")?
The CI could fail for a number of reasons.
At the bottom of the pull request, where it says whether your build passed or failed, you can click “Details” next to the test, which takes you to the Travis page.
You can view the log or rerun the checks if you have write access to the repo by clicking the “Restart build” button in the top right (you must be logged in to Travis CI with your GitHub account see the “Restart build” button).
At the bottom of the pull request, where it says whether your build passed or failed, you can click “Details” next to the test, which takes you to the [GitHub Actions](https://github.com/urbangrammarai/gee_pipeline/actions) page. From there you can view the log or rerun the checks (if you have write-access to the repo).

GitHub has a [nice introduction][github-flow] to the pull request workflow, but please get in touch if you have any questions.

Expand Down
2 changes: 1 addition & 1 deletion LICENSE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ Permission is hereby granted, free of charge, to any person obtaining a copy of

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1 +1 @@
include pyveg/testdata/*
include pyveg/testdata/*
2 changes: 1 addition & 1 deletion binder/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ earthengine-api==0.1.210
geetools
pillow
imageio
opencv-python
opencv-python
45 changes: 23 additions & 22 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,66 +12,67 @@
#
import os
import sys
sys.path.insert(0, os.path.abspath('../../pyveg'))

sys.path.insert(0, os.path.abspath("../../pyveg"))

# -- Project information -----------------------------------------------------

project = 'pyveg'
copyright = '2020, Nick Barlow, Camila Rangel Smith, Samuel Van Stroud, Jesse F. Abrams, Chris Boulton, Joshua Buxton.'
author = 'Nick Barlow, Camila Rangel Smith, Samuel Van Stroud, Jesse F. Abrams, Chris Boulton, Joshua Buxton.'
project = "pyveg"
copyright = "2020, Nick Barlow, Camila Rangel Smith, Samuel Van Stroud, Jesse F. Abrams, Chris Boulton, Joshua Buxton."
author = "Nick Barlow, Camila Rangel Smith, Samuel Van Stroud, Jesse F. Abrams, Chris Boulton, Joshua Buxton."

# The full version, including alpha/beta/rc tags
release = '1.0.0'
release = "1.0.0"


# -- General configuration ---------------------------------------------------
master_doc = 'index'
master_doc = "index"

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.autosummary',
'sphinx.ext.doctest',
'sphinx.ext.intersphinx',
'sphinx.ext.todo',
'sphinx.ext.coverage',
'sphinx.ext.mathjax',
'sphinx.ext.napoleon',
'sphinx.ext.viewcode',
'recommonmark'
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"sphinx.ext.doctest",
"sphinx.ext.intersphinx",
"sphinx.ext.todo",
"sphinx.ext.coverage",
"sphinx.ext.mathjax",
"sphinx.ext.napoleon",
"sphinx.ext.viewcode",
"recommonmark",
]

napoleon_use_ivar = True

# for Sphinx-1.3

source_suffix = ['.rst', '.md']
source_suffix = [".rst", ".md"]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
templates_path = ["_templates"]

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = 'y'
language = "y"

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'
html_theme = "sphinx_rtd_theme"

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
html_static_path = ["_static"]
9 changes: 9 additions & 0 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name: gee_pipeline
channels:
- defaults
- conda-forge
dependencies:
- python=3.7
- earthengine-api
- rasterio
- coverage
Loading