Skip to content

Commit

Permalink
more detail
Browse files Browse the repository at this point in the history
  • Loading branch information
andygrove committed Jun 12, 2024
1 parent 7128326 commit 5e89f64
Show file tree
Hide file tree
Showing 3 changed files with 82 additions and 22 deletions.
85 changes: 64 additions & 21 deletions dev/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,35 +92,78 @@ Create a PR against the main branch to update the Rust crate version to `0.2.0`

This part of the process can mostly only be performed by a PMC member.

- Run the create-tarball script on the release candidate tag (`0.1.0-rc1`) to create the source tarball and upload it to the dev subversion repository
- Start an email voting thread
- Once the vote passes, run the release-tarball script to move the tarball to the release subversion repository
- Register the release with the [Apache Reporter Service](https://reporter.apache.org/addrelease.html?datafusion) using
a version such as `COMET-0.1.0`
- Delete old release candidates and releases from the subversion repositories
- Push a release tag (`0.1.0`) to the Apache repository
- Reply to the vote thread to close the vote and announce the release
### Create the Release Candidate Tarball

## Verifying Release Candidates
Run the create-tarball script on the release candidate tag (`0.1.0-rc1`) to create the source tarball and upload it to the dev subversion repository

The vote email will link to this section of this document, so this is where we will need to provide instructions for
verifying a release candidate.
```shell
GH_TOKEN=<TOKEN> ./dev/release/create-tarball.sh 0.1.0 1
```

### Start an Email Voting Thread

Send the email that is generated in the previous step to `[email protected]`.

### Publish the Release Tarball

The `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification
process. It checks the hashes and runs the build. It does not run the test suite because this takes a long time
for this project and the test suites already run in CI before we create the release candidate, so running them
again is somewhat redundant.
Once the vote passes, run the release-tarball script to move the tarball to the release subversion repository.

```shell
./dev/release/verify-release-candidate.sh 0.1.0 1
./dev/release/create-tarball.sh 0.1.0 1
```

We hope that users will verify the release beyond running this script by testing the release candidate with their
existing Spark jobs and report any functional issues or performance regressions.
Push a release tag (`0.1.0`) to the Apache repository.

```shell
git fetch apache
git checkout 0.1.0-rc1
git tag 0.1.0
git push apache 0.1.0
```

Reply to the vote thread to close the vote and announce the release.

## Post Release Admin

Register the release with the [Apache Reporter Service](https://reporter.apache.org/addrelease.html?datafusion) using
a version such as `COMET-0.1.0`.

### Delete old RCs and Releases

See the ASF documentation on [when to archive](https://www.apache.org/legal/release-policy.html#when-to-archive)
for more information.

#### Deleting old release candidates from `dev` svn

Release candidates should be deleted once the release is published.

Get a list of DataFusion Comet release candidates:

Another way of verifying the release is to follow the
[Comet Benchmarking Guide](https://datafusion.apache.org/comet/contributor-guide/benchmarking.html) and compare
performance with the previous release.
```shell
svn ls https://dist.apache.org/repos/dist/dev/datafusion | grep comet
```

Delete a release candidate:

```shell
svn delete -m "delete old DataFusion Comet RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.1.0-rc1/
```

#### Deleting old releases from `release` svn

Only the latest release should be available. Delete old releases after publishing the new release.

Get a list of DataFusion releases:

```shell
svn ls https://dist.apache.org/repos/dist/release/datafusion | grep comet
```

Delete a release:

```shell
svn delete -m "delete old DataFusion Comet release" https://dist.apache.org/repos/dist/release/datafusion-comet/datafusion-comet-0.0.0
```

## Publishing Binary Releases

Expand Down
2 changes: 1 addition & 1 deletion dev/release/create-tarball.sh
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ on the release. The vote will be open for at least 72 hours.
Only votes from PMC members are binding, but all members of the community are
encouraged to test the release and vote with "(non-binding)".
The standard verification procedure is documented at https://github.com/apache/datafusion-comet/blob/main/dev/release/README.md#verifying-release-candidates.
The standard verification procedure is documented at https://github.com/apache/datafusion-comet/blob/main/dev/release/verifying-release-candidates.md
[ ] +1 Release this as Apache DataFusion Comet ${version}
[ ] +0
Expand Down
17 changes: 17 additions & 0 deletions dev/release/verifying-release-candidates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Verifying DataFusion Comet Release Candidates

The `dev/release/verify-release-candidate.sh` script in this repository can assist in the verification
process. It checks the hashes and runs the build. It does not run the test suite because this takes a long time
for this project and the test suites already run in CI before we create the release candidate, so running them
again is somewhat redundant.

```shell
./dev/release/verify-release-candidate.sh 0.1.0 1
```

We hope that users will verify the release beyond running this script by testing the release candidate with their
existing Spark jobs and report any functional issues or performance regressions.

Another way of verifying the release is to follow the
[Comet Benchmarking Guide](https://datafusion.apache.org/comet/contributor-guide/benchmarking.html) and compare
performance with the previous release.

0 comments on commit 5e89f64

Please sign in to comment.