Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/develop' into issue_3114_check…
Browse files Browse the repository at this point in the history
…_coord
  • Loading branch information
RuiheLi committed Nov 27, 2023
2 parents 68a3b77 + 6ece7a1 commit c906728
Show file tree
Hide file tree
Showing 260 changed files with 9,436 additions and 8,554 deletions.
36 changes: 33 additions & 3 deletions .all-contributorsrc
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,7 @@
"login": "wigging",
"name": "Gavin Wiggins",
"avatar_url": "https://avatars.githubusercontent.com/u/6828967?v=4",
"profile": "https://wigging.me",
"profile": "https://gavinw.me",
"contributions": [
"bug",
"code"
Expand Down Expand Up @@ -531,7 +531,8 @@
"contributions": [
"infra",
"code",
"doc"
"doc",
"review"
]
},
{
Expand Down Expand Up @@ -613,7 +614,8 @@
"contributions": [
"infra",
"code",
"doc"
"doc",
"review"
]
},
{
Expand Down Expand Up @@ -697,6 +699,34 @@
"code",
"test"
]
},
{
"login": "aitorres",
"name": "Andrés Ignacio Torres",
"avatar_url": "https://avatars.githubusercontent.com/u/26191851?v=4",
"profile": "https://aitorres.com",
"contributions": [
"infra"
]
},
{
"login": "Agnik7",
"name": "Agnik Bakshi",
"avatar_url": "https://avatars.githubusercontent.com/u/77234005?v=4",
"profile": "https://github.com/Agnik7",
"contributions": [
"doc"
]
},
{
"login": "RuiheLi",
"name": "RuiheLi",
"avatar_url": "https://avatars.githubusercontent.com/u/84007676?v=4",
"profile": "https://github.com/RuiheLi",
"contributions": [
"code",
"test"
]
}
],
"contributorsPerLine": 7,
Expand Down
3 changes: 2 additions & 1 deletion .github/release_reminder.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
title: Create {{ date | date('YY.MM') }} (final or rc0) release
labels: priority:high
---
Quarterly reminder to create a -

1. pre-release if the month has just started.
2. non-pre-release if the month is about to end (**before the end of the month**).

See [Release Workflow](./release_workflow.md) for more information.
See [Release Workflow](https://github.com/pybamm-team/PyBaMM/blob/develop/.github/release_workflow.md) for more information.
38 changes: 22 additions & 16 deletions .github/release_workflow.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,44 @@
# Release workflow

This file contains the workflow required to make a `PyBaMM` release on GitHub and PyPI by the maintainers.
This file contains the workflow required to make a `PyBaMM` release on GitHub, PyPI, and conda-forge by the maintainers.

## rc0 releases (automated)

1. The `update_version.yml` workflow will run on every 1st of January, May and September, updating incrementing the version to `YY.MMrc0` by running `scripts/update_version.py` in the following files -
1. The `update_version.yml` workflow will run on every 1st of January, May and September, updating incrementing the version to `vYY.MMrc0` by running `scripts/update_version.py` in the following files -

- `pybamm/version.py`
- `docs/conf.py`
- `CITATION.cff`
- `pyproject.toml`
- `vcpkg.json`
- `docs/_static/versions.json`
- `CHANGELOG.md`

These changes will be automatically pushed to a new branch `YY.MM`.
These changes will be automatically pushed to a new branch `vYY.MM` and a PR from `vvYY.MM` to `develop` will be created (to sync the branches).

2. Create a new GitHub _pre-release_ with the tag `YY.MMrc0` from the `YY.MM` branch and a description copied from `CHANGELOG.md`.
2. Create a new GitHub _pre-release_ with the tag `vYY.MMrc0` from the `vYY.MM` branch and a description copied from `CHANGELOG.md`.

3. This release will automatically trigger `publish_pypi.yml` and create a _pre-release_ on PyPI.

## rcX releases (manual)

If a new release candidate is required after the release of `rc0` -

1. Fix a bug in `YY.MM` (no new features should be added to `YY.MM` once `rc0` is released) and `develop` individually.
1. Fix a bug in `vYY.MM` (no new features should be added to `vYY.MM` once `rc0` is released) and `develop` individually.

2. Run `update_version.yml` manually while using `append_to_tag` to specify the release candidate version number (`rc1`, `rc2`, ...).

3. This will increment the version to `YY.MMrcX` by running `scripts/update_version.py` in the following files -
3. This will increment the version to `vYY.MMrcX` by running `scripts/update_version.py` in the following files -

- `pybamm/version.py`
- `docs/conf.py`
- `CITATION.cff`
- `pyproject.toml`
- `vcpkg.json`
- `docs/_static/versions.json`
- `CHANGELOG.md`

These changes will be automatically pushed to the existing branch `YY.MM`.
These changes will be automatically pushed to the existing `vYY.MM` branch and a PR from `vvYY.MM` to `develop` will be created (to sync the branches).

4. Create a new GitHub _pre-release_ with the same tag (`YY.MMrcX`) from the `YY.MM` branch and a description copied from `CHANGELOG.md`.
4. Create a new GitHub _pre-release_ with the same tag (`vYY.MMrcX`) from the `vYY.MM` branch and a description copied from `CHANGELOG.md`.

5. This release will automatically trigger `publish_pypi.yml` and create a _pre-release_ on PyPI.

Expand All @@ -48,18 +48,18 @@ Once satisfied with the release candidates -

1. Run `update_version.yml` manually, leaving the `append_to_tag` field blank ("") for an actual release.

2. This will increment the version to `YY.MMrcX` by running `scripts/update_version.py` in the following files -
2. This will increment the version to `vYY.MMrcX` by running `scripts/update_version.py` in the following files -

- `pybamm/version.py`
- `docs/conf.py`
- `CITATION.cff`
- `pyproject.toml`
- `vcpkg.json`
- `docs/_static/versions.json`
- `CHANGELOG.md`

These changes will be automatically pushed to the existing branch `YY.MM`.
These changes will be automatically pushed to the existing `vYY.MM` branch and a PR from `vvYY.MM` to `develop` will be created (to sync the branches).

3. Next, a PR from `YY.MM` to `main` will be generated that should be merged once all the tests pass.
3. Next, a PR from `vYY.MM` to `main` will be generated that should be merged once all the tests pass.

4. Create a new GitHub _release_ with the same tag from the `main` branch and a description copied from `CHANGELOG.md`.

Expand All @@ -70,5 +70,11 @@ Once satisfied with the release candidates -
Some other essential things to check throughout the release process -

- If updating our custom vcpkg registory entries [pybamm-team/sundials-vcpkg-registry](https://github.com/pybamm-team/sundials-vcpkg-registry) or [pybamm-team/casadi-vcpkg-registry](https://github.com/pybamm-team/casadi-vcpkg-registry) (used to build Windows wheels), make sure to update the baseline of the registories in vcpkg-configuration.json to the latest commit id.
- Update jax and jaxlib to the latest version in `pybamm.util` and `setup.py`, fixing any bugs that arise
- Make sure the URLs in `docs/_static/versions.json` are valid
- Update jax and jaxlib to the latest version in `pybamm.util` and `pyproject.toml`, fixing any bugs that arise
- As the release workflow is initiated by the `release` event, it's important to note that the default `GITHUB_REF` used by `actions/checkout` during the checkout process will correspond to the tag created during the release process. Consequently, the workflows will consistently build PyBaMM based on the commit associated with this tag. Should new commits be introduced to the `vYY.MM` branch, such as those addressing build issues, it becomes necessary to manually update this tag to point to the most recent commit -
```
git tag -f <tag_name> <commit_hash>
git push -f <pybamm-team/PyBaMM_remote_name> <tag_name> # can only be carried out by the maintainers
```
- If changes are made to the API, console scripts, entry points, new optional dependencies are added, support for major Python versions is dropped or added, or core project information and metadata are modified at the time of the release, make sure to update the `meta.yaml` file in the `recipe/` folder of the [conda-forge/pybamm-feedstock](https://github.com/conda-forge/pybamm-feedstock) repository accordingly by following the instructions in the [conda-forge documentation](https://conda-forge.org/docs/maintainer/updating_pkgs.html#updating-the-feedstock-repository) and re-rendering the recipe
- The conda-forge release workflow will automatically be triggered following a stable PyPI release, and the aforementioned updates should be carried out directly in the main repository by pushing changes to the automated PR created by the conda-forge-bot. A manual PR can also be created if the updates are not included in the automated PR for some reason. This manual PR **must** bump the build number in `meta.yaml` and **must** be from a personal fork of the repository.
6 changes: 6 additions & 0 deletions .github/wheel_failure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
title: Fortnightly build for wheels failed
labels: priority:high, bug
---

The build is failing with the following logs - {{ env.LOGS }}
13 changes: 9 additions & 4 deletions .github/workflows/benchmark_on_push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,20 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: 3.8

- name: Install Linux system dependencies
run: |
sudo apt-get update
sudo apt install gfortran gcc libopenblas-dev
- name: Install python dependencies
# Pin asv==0.5.1 to fix failing benchmarks. Related to https://github.com/airspeed-velocity/asv/issues/1323
run: |
python -m pip install --upgrade pip wheel setuptools virtualenv asv==0.5.1 wget cmake casadi numpy
- name: Install SuiteSparse and Sundials
python -m pip install --upgrade pip wheel setuptools wget cmake casadi numpy
python -m pip install asv[virtualenv]
- name: Install SuiteSparse and SUNDIALS
run: python scripts/install_KLU_Sundials.py

- name: Fetch base branch
run: |
# This workflow also runs for merge commits
Expand All @@ -49,7 +53,8 @@ jobs:
HEAD_COMMIT=$(git rev-parse HEAD)
echo $BASE_COMMIT | tee commits_to_compare.txt
echo $HEAD_COMMIT | tee -a commits_to_compare.txt
asv run HASHFILE:commits_to_compare.txt --m "GitHubRunner" --show-stderr --strict -v
asv run HASHFILE:commits_to_compare.txt --m "GitHubRunner" --show-stderr -v
- name: Compare commits' benchmark results
run: |
BASE_COMMIT=$(head -1 commits_to_compare.txt)
Expand Down
86 changes: 86 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: Build and push Docker images to Docker Hub

on:
workflow_dispatch:
push:
branches:
- develop

jobs:
build_docker_images:
# This workflow is only of value to PyBaMM and would always be skipped in forks
if: github.repository_owner == 'pybamm-team'
name: Image (${{ matrix.build-args }})
runs-on: ubuntu-latest
strategy:
matrix:
build-args: ["No solvers", "JAX", "ODES", "IDAKLU", "ALL"]
fail-fast: true

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Create tags for Docker images based on build-time arguments
id: tags
run: |
if [ "${{ matrix.build-args }}" = "No solvers" ]; then
echo "tag=latest" >> "$GITHUB_OUTPUT"
elif [ "${{ matrix.build-args }}" = "JAX" ]; then
echo "tag=jax" >> "$GITHUB_OUTPUT"
elif [ "${{ matrix.build-args }}" = "ODES" ]; then
echo "tag=odes" >> "$GITHUB_OUTPUT"
elif [ "${{ matrix.build-args }}" = "IDAKLU" ]; then
echo "tag=idaklu" >> "$GITHUB_OUTPUT"
elif [ "${{ matrix.build-args }}" = "ALL" ]; then
echo "tag=all" >> "$GITHUB_OUTPUT"
fi
- name: Build and push Docker image to Docker Hub (no solvers)
if: matrix.build-args == 'No solvers'
uses: docker/build-push-action@v5
with:
context: .
file: scripts/Dockerfile
tags: pybamm/pybamm:${{ steps.tags.outputs.tag }}
push: true
platforms: linux/amd64, linux/arm64

- name: Build and push Docker image to Docker Hub (with ODES and IDAKLU solvers)
if: matrix.build-args == 'ODES' || matrix.build-args == 'IDAKLU'
uses: docker/build-push-action@v5
with:
context: .
file: scripts/Dockerfile
tags: pybamm/pybamm:${{ steps.tags.outputs.tag }}
push: true
build-args: ${{ matrix.build-args }}=true
platforms: linux/amd64, linux/arm64

- name: Build and push Docker image to Docker Hub (with ALL and JAX solvers)
if: matrix.build-args == 'ALL' || matrix.build-args == 'JAX'
uses: docker/build-push-action@v5
with:
context: .
file: scripts/Dockerfile
tags: pybamm/pybamm:${{ steps.tags.outputs.tag }}
push: true
build-args: ${{ matrix.build-args }}=true
# exclude arm64 for JAX and ALL builds for now, see
# https://github.com/google/jax/issues/13608
platforms: linux/amd64

- name: List built image(s)
run: docker images
5 changes: 5 additions & 0 deletions .github/workflows/lychee_url_checker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,18 @@ jobs:
--accept 200,429
--exclude-path ./CHANGELOG.md
--exclude-path ./scripts/update_version.py
--exclude-path asv.conf.json
--exclude-path docs/conf.py
'./**/*.rst'
'./**/*.md'
'./**/*.py'
'./**/*.ipynb'
'./**/*.json'
'./**/*.toml'
# fail the action on broken links
fail: true
jobSummary: true
format: markdown
env:
# to be used in case rate limits are surpassed
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
18 changes: 15 additions & 3 deletions .github/workflows/periodic_benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,26 +20,33 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python 3.8
uses: actions/setup-python@v4
with:
python-version: 3.8

- name: Install Linux system dependencies
run: |
sudo apt-get update
sudo apt-get install gfortran gcc libopenblas-dev
- name: Install python dependencies
run: |
python -m pip install --upgrade pip wheel setuptools virtualenv asv wget cmake casadi numpy
- name: Install SuiteSparse and Sundials
python -m pip install --upgrade pip wheel setuptools wget cmake casadi numpy
python -m pip install asv[virtualenv]
- name: Install SuiteSparse and SUNDIALS
run: python scripts/install_KLU_Sundials.py

- name: Run benchmarks
run: |
asv machine --machine "GitHubRunner"
asv run --machine "GitHubRunner" NEW --show-stderr --strict -v
asv run --machine "GitHubRunner" NEW --show-stderr -v
env:
SUNDIALS_INST: $HOME/.local
LD_LIBRARY_PATH: $HOME/.local/lib

- name: Upload results as artifact
uses: actions/upload-artifact@v3
with:
Expand All @@ -55,18 +62,22 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: 3.8

- name: Install asv
run: pip install asv

- name: Checkout pybamm-bench repo
uses: actions/checkout@v4
with:
repository: pybamm-team/pybamm-bench
token: ${{ secrets.BENCH_PAT }}

- name: Download results artifact
uses: actions/download-artifact@v3
with:
name: asv_new_results
path: new_results

- name: Copy new results and push to pybamm-bench repo
env:
PUSH_BENCH_EMAIL: ${{ secrets.PUSH_BENCH_EMAIL }}
Expand All @@ -78,6 +89,7 @@ jobs:
git add results
git commit -am "Add new results"
git push
- name: Publish results
run: |
asv publish
Expand Down
Loading

0 comments on commit c906728

Please sign in to comment.