Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new JSON output: supported Python versions #102

Merged
merged 13 commits into from
Mar 27, 2024
75 changes: 75 additions & 0 deletions .github/workflows/ci-supported-pythons.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
name: Use JSON output to build job matrix

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

env:
FORCE_COLOR: "1" # Make tools pretty.

jobs:
build-package:
name: Build & verify package
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
repository: hynek/structlog
path: structlog
- uses: actions/checkout@v4
with:
path: action
- uses: ./action
id: baipp
with:
path: structlog

outputs:
python-versions: ${{ steps.baipp.outputs.supported_python_classifiers_json_array }}
# If your matrix consists only of Python versions, you can use the
# following, too:
# python-versions: ${{ steps.baipp.outputs.supported_python_classifiers_json_job_matrix_value }}

test-package:
needs: build-package
runs-on: ubuntu-latest
strategy:
matrix:
# Create matrix from the 'python-versions' output from the build-package
# job.
python-version: ${{ fromJson(needs.build-package.outputs.python-versions) }}

# If you set 'python-versions' to
# 'supported_python_classifiers_json_job_matrix_value'
# above, you would set the matrix like this instead:
# matrix: ${{ fromJson(needs.build-package.outputs.python-versions) }}

steps:
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Download built packages from the build-package job.
uses: actions/download-artifact@v4
with:
name: Packages
path: dist

- name: Prepare tests & config
run: |
# We use tox together with the fast tox-uv plugin.
python -Im pip install tox-uv

# Unpack SDist for tests & config files.
tar xf dist/*.tar.gz --strip-components=1

# Ensure tests run against wheel.
rm -rf src

- run: python -Im tox run --installpkg dist/*.whl -f py$(echo ${{ matrix.python-version }} | tr -d .)

...
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),

### Added

- New outputs: `supported_python_classifiers_json_array` and `supported_python_classifiers_json_job_matrix_value`.

They are extracted from the trove classifiers defined in the package metadata (for example, `Programming Language :: Python :: 3.12`) and allow you to define the Python versions matrix for your CI jobs without duplicating this information.
[#80](https://github.com/hynek/build-and-inspect-python-package/pull/80)
[#102](https://github.com/hynek/build-and-inspect-python-package/pull/102)

- New input: `skip-wheel` to skip building the wheel in addition to the source distribution.
This is useful if you need to build your wheels using advanced tools like [*cibuildwheel*](https://cibuildwheel.pypa.io/) anyway.
[#98](https://github.com/hynek/build-and-inspect-python-package/pull/98)
Expand Down
39 changes: 30 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

*build-and-inspect-python-package* is a GitHub Action that provides the following functionality to Python package maintainers:

**Builds your package** using PyPA's [*build*](https://pypi.org/project/build/) (this works with any [PEP 517](https://peps.python.org/pep-0517/)-compatible build backend, including Hatch, Flit, Setuptools, PDM, or Poetry).
**Builds your package** using PyPAs [*build*](https://pypi.org/project/build/) (this works with any [PEP 517](https://peps.python.org/pep-0517/)-compatible build backend, including Hatch, Flit, Setuptools, PDM, or Poetry).
[`SOURCE_DATE_EPOCH`](https://reproducible-builds.org/specs/source-date-epoch/) is set to the timestamp of the last commit, giving you reproducible builds with meaningful file timestamps.

Uploads the **built *wheel* and the source distribution (*SDist*) as GitHub Actions artifacts**, so you can download and inspect them from the Summary view of a run, or [**upload them to PyPI automatically**][automated] once the verification succeeds.
Expand All @@ -16,7 +16,7 @@ Lints the **wheel contents** using [*check-wheel-contents*](https://pypi.org/pro
Lints the **PyPI README** using [Twine](https://pypi.org/project/twine/) and uploads it as an GitHub Actions artifact for further manual inspection.
To level up your PyPI README game, check out [*hatch-fancy-pypi-readme*](https://github.com/hynek/hatch-fancy-pypi-readme)!

Prints the **tree of both *SDist* and *wheel*** in the CI output, so you don't have to download the packages, if you just want to check the content list.
Prints the **tree of both *SDist* and *wheel*** in the CI output, so you dont have to download the packages, if you just want to check the content list.

Prints and uploads the **packaging metadata** as a GitHub Actions artifact.

Expand All @@ -43,9 +43,16 @@ It uploads every commit on `main` to [Test PyPI](https://test.pypi.org/project/s
This is easily achievable using tools like [*setuptools-scm*](https://setuptools-scm.readthedocs.io/) or [*hatch-vcs*](https://github.com/ofek/hatch-vcs), but beyond the scope of this humble README.


### Define Python Version Matrix Based On Package Metadata

*build-and-inspect-python-package* extracts the Python versions your package supports from the trove classifiers in your package’s metadata and offers them as an action output.

That means that you can define your CI matrix based on the Python versions your package supports without duplicating the information between your package configuration and your CI configuration.


### Applications

If you package an **application** as a Python package, this action is useful to double-check you're shipping everything you need, including all templates, translation files, et cetera.
If you package an **application** as a Python package, this action is useful to double-check youre shipping everything you need, including all templates, translation files, et cetera.


## Usage
Expand Down Expand Up @@ -92,26 +99,39 @@ While *build-and-inspect-python-package* will build a wheel for you by default,

### Outputs

- `dist`: the location with the built packages.
- `dist`: The location with the built packages.

See, for example, how [*argon2-cffi-bindings*](https://github.com/hynek/argon2-cffi-bindings/blob/daff9ceb693312ab8257c60db4cd1c13cd866a35/.github/workflows/ci.yml#L83-L97) uses this feature to check the built wheels don’t break a package that depends on it.

See, for example, how [*argon2-cffi-bindings*](https://github.com/hynek/argon2-cffi-bindings/blob/daff9ceb693312ab8257c60db4cd1c13cd866a35/.github/workflows/ci.yml#L83-L97) uses this feature to check the built wheels don't break a package that depends on it.
- `supported_python_classifiers_json_array`: A JSON array of Python versions that are supported by the package as defined by the trove classifiers in the package metadata (for example, `Programming Language :: Python :: 3.12`).

You can assign this to a matrix strategy key in your CI job (for example, `strategy.matrix.python-version`) to test against multiple Python versions without duplicating the information.
Since GitHub Actions only allows for strings as variables, you have to parse it with `fromJSON` in your workflow.

If all this sounds confusing: Check out our [supported Pythons CI workflow] for a realistic example.

- `supported_python_classifiers_json_job_matrix_value`: Same as `supported_python_classifiers_json_array`, but it’s a mapping with the JSON array bound to the `python-version` key.

This is useful if you only want to define a matrix based on Python versions, because then you can just assign this to `strategy.matrix`.


### Artifacts

After a successful run, you'll find multiple artifacts in the run's Summary view:
After a successful run, youll find the following artifacts in the runs Summary view:

- **Packages**: The built packages.
Perfect for [automated PyPI upload workflows][automated]!
- **Package Metadata**: the extracted packaging metadata (*hint*: it's formatted as an email).
- **PyPI README**: the extracted PyPI README, exactly how it would be used by PyPI as your project's landing page.
[PEP 621](https://peps.python.org/pep-0621/) calls it `readme`, in classic *setuptools* it's `long_description`.
- **Package Metadata**: the extracted packaging metadata (*hint*: its formatted as an email).
- **PyPI README**: the extracted PyPI README, exactly how it would be used by PyPI as your projects landing page.
[PEP 621](https://peps.python.org/pep-0621/) calls it `readme`, in classic *setuptools* its `long_description`.


### Examples

[Our CI](.github/workflows/ci.yml) uses all inputs and outputs, if you want to see them in action.

Our [supported Pythons CI workflow] demonstrates how to use `supported_python_classifiers_json_array` to set up a matrix of Python versions for your CI jobs without duplicating the information with your packaging metadata.


## License

Expand All @@ -120,3 +140,4 @@ The scripts and documentation in this project are released under the [MIT Licens
[automated]: https://github.com/python-attrs/attrs/blob/main/.github/workflows/pypi-package.yml
[*cibuildwheel*]: https://cibuildwheel.pypa.io/
[*setuptools-scm*]: https://setuptools-scm.readthedocs.io/
[supported Pythons CI workflow]: .github/workflows/ci-supported-pythons.yml
35 changes: 33 additions & 2 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,21 @@ inputs:
outputs:
dist:
description: The location of the built packages.
value: ${{ steps.setter.outputs.dist }}
value: ${{ steps.dist-location-setter.outputs.dist }}
supported_python_classifiers_json_array:
description: >
A JSON array that contains all classifier-declared supported Python
versions. When loaded using the 'fromJson' function, this can be assigned
to a matrix strategy key (for example, `python-version`).

value: ${{ steps.supported-pythons-setter.outputs.supported_python_classifiers_json_array }}
supported_python_classifiers_json_job_matrix_value:
description: >
Same as 'supported_python_classifiers_json_array', except it's already a
JSON mapping from "python-version" to a list of all classifier-declared
supported Python versions. In other words, you can assign it directly to
the 'strategy.matrix' key.
value: ${{ steps.supported-pythons-setter.outputs.supported_python_classifiers_json_job_matrix_value }}

runs:
using: composite
Expand Down Expand Up @@ -78,7 +92,7 @@ runs:
working-directory: ${{ inputs.path }}

- name: Set output
id: setter
id: dist-location-setter
shell: bash
run: echo "dist=/tmp/baipp/dist" >>$GITHUB_OUTPUT

Expand Down Expand Up @@ -165,3 +179,20 @@ runs:
with:
name: PyPI README${{ inputs.upload-name-suffix }}
path: /tmp/baipp/dist/out/sdist/PyPI-README.*

- name: Generate JSON objects of supported Python versions
id: supported-pythons-setter
shell: bash
working-directory: /tmp/baipp/dist/out/sdist/
run: |
cat */PKG-INFO | python -c '
import json, re, sys
match_classifier = re.compile(
r"\s*Classifier: Programming Language :: Python :: (\d+\.\d+)$"
).match
version_tokens = [
m.group(1).strip() for l in sys.stdin.readlines() if (m := match_classifier(l))
]
print(f"supported_python_classifiers_json_array={json.dumps(version_tokens)}")
print(f"""supported_python_classifiers_json_job_matrix_value={json.dumps({"python-version": version_tokens})}""")
' >> $GITHUB_OUTPUT