Proposal for R package release process guidelines

[zkamvar]

Note: this is of most interest to members of the hubverse developer team and any contributors. Users of the hubverse are welcome to comment, but ultimately the decision will be made by the development team. I am most interested in the following:

1. how you feel about the proposed workflow on a 1 to 3 scale a la Martha's Rules: (1: like it, 2: no strong opinions, 3: dislike it).
2. what clarifying questions do you have?

(also sorry for the length, TL;DR: let's use tags for releases 🏷️ 📦)

Background: All around the R-Universe 🛰️ ☄️

A couple of weeks ago, I enabled the creation of https://hubverse-org.r-universe.dev/. The R-universe is a CRAN-like repository that will check for updates to the GitHub versions of the packages and build them to binary versions on an hourly basis.

I wrote an rOpenSci blog post (8-10 minute read) about the advantages of the R-universe, but one major advantage is that users do not have to install packages using the {remotes} package, which means that they can use one line to install binary packages for their system:

install.packages("hubverse", repos = c("https://hubverse-org.r-universe.dev", "https://cran.r-project.org"))

Aside from the user advantages, this means we can now be more intentional with package releases.

Current release process: main as release

As of now, all packages are released from the main branch. This means that every merged pull request is considered a release.

Advantages:

• conceptually straightforward 😃👍🏽

Disadvantages:

• large pull requests with disparate features (harder to review)
• pull requests pointing to pull requests
• hard to perform integration tests across packages before they are released
• easier for bugs to find their way to users (so hotfixes are more common)
• typo fixes require new versions (note: not necessarily a bad thing, just a thing)

Proposal: buffered release from tag 🏷️

I would like to propose to use tags and GitHub releases to officially manage releases. This would effectively make the main branch the "release candidate" version to add incremental changes.

Last year, I wrote up a detailed document on the release process for The Carpentries Workbench that describes this process.

In short, the process of releasing an R package looks like this:

1. pull requests go to the main branch as usual
   i. Iterate.
   ii. Test. For people comfortable with testing, have them use remotes::install_github("huberse/{pkg}") to test it locally
2. when you are ready to release,
   i. bump the version number
   ii. create a (preferably signed) tag X.Y.Z and do a GitHub release from that tag, summarizing the changes in the comments
3. bump the version number in main to a development (X.Y.Z.9000) version

Advantages:

• We have a stronger assurance of quality via the second step of adding a tag 💪🏽😤👍🏽
• Easy to install older versions if needed ⏪
• Easy to compare changes between versions 🔍👀
• Can create integration tests to confirm that cross-package changes work ✅
• Reduced size and complexity of pull requests because we can split them into single-topics 1️⃣
• Easier to create workflows around release announcements 😃📣

Disadvantages:

• Hotfixes require branching off of a tag. The workflow is slightly more complex than a pull request workflow and I have a writeup on how to create hotfixes from tags in 4 steps. For those who are uncomfortable with git, it can feel like trying to drink water from the wrong side of a cup the first few times.

Documenting the workflow 📝

Whatever we decide for the official workflow, we should document it somewhere. @annakrystalli has suggested as a vignette in hubDevs.

Related: Release schedule 🗓️

At the moment, we do not have a release schedule (as far as I know). I have been fairly terrible about this in my own work, but if we are trying to build and sustain a community, setting expectations like this reduces uncertainty and surprise and it gives users a way to prepare for, test, and even get excited about upcoming changes. This is not really something that we can do with our current workflow.

[nickreich]

I'm a "1" on this (like it)!

Thanks for the detailed run-down.

My one question: what is a "signed" tag?

[zkamvar]

Signing a commit or tag is a way of using an SSH or GPG key to verify that the commit actually came from your machine, as opposed to someone appearing to be you from a different machine. Whenever you make a commit through GitHub's web interface, this is applied automatically, but you have to explicitly configure it to add the signature on your computer.

For example, all of my commits are signed, so they all get a green "verified" tag next to them, which you can use to check that I was the one who actually made the commit.

Example of unsigned commits not coming from original author

A very real example of this are commits that have identical content in the Instructor Training repository for The Carpentries:

carpentries/instructor-training@8374b76
carpentries/instructor-training@3ba6c7f

Both of these have the same author, same content, and same timestamp, but only the first one has a signed commit (albiet expired). This is because the first commit is the only one that user actually made. The second commit was programmatically made by me when I was transitioning all Carpentries lessons away from Jekyll.

[annakrystalli]

1 from me (actually more like +💯😜)

[bsweger]

Love it-thanks for the proposal!

Reduced size and complexity of pull requests because we can split them into single-topics 1️⃣

This is a huge advantage to what you're proposing (which is akin to a "dev branch" convo we had a while back, albeit more structured--we need a landing place for pre-release commits).

Another advantage is making it easier for more than one person at a time to work in the code base, so big 👍 from me! Would love to discuss doing the same for our Python code 😄

[matthewcornell]

A 1 from me too, with the disclaimer that most of my work has been solo, so detailed workflows are fairly new to me. Thanks for the proposal.

[zkamvar]

Thank you! Since all of the hubverse repos already have the default branch as a protected branch (i.e. you can't push directly to it) and all contributions are made via pull request, you probably won't notice too much of a difference other than the PRs become more atomic. The only extra step is adding the tag and release.

[elray1]

I am a 1.5: I don't really have strong opinions about this, but happy to go with this proposal. I like the idea of laying out formal process. I think the only thing I would have proposed differently would be keeping the release candidate on a branch other than main until we're ready to do the release, so that people installing from github would get the latest release by default. But this concern is made somewhat obsolete if we provide installation instructions that point to r-universe (or cran). And, again, I'm happy to go with the proposal here!

[zkamvar]

I realized I never put in a response to your concern. Sorry about that.

In practice, both scenarios would lead to the benefits of smaller pull requests, clear release notes with an easily traceable history and no longer needing to create pull requests into other pull requests. Both of them have the ability to give you the latest release from GitHub with the @*release tag
(e.g. remotes::install_github("hubverse-org/hubAdmin@*release"))

If we consider the two scenarios, there are different tradeoffs:

main is identical to release

• pro: remotes::install_github() and install.packages() effectively identical, so people do not have to think about where to install from.
• con: developers would need to maintain a separate devel-specific branch, with the following implications:
  • this branch cannot be the default branch or you don't get the advantage of identical versions from GitHub and R-universe.
  • 🤕 cognitive load (every time): developers would need to remember to open PRs to this branch every time
  • workflows need to be rewritten to include this branch
    • amelioration: this needs only be done once

main is active stable development

• pro: There is a clear distinction between "released" (on CRAN/R-universe) vs. "development" (on GitHub)
• pro: Developers do not need to change their workflow much to accommodate normal scenario-type work.
• con: People who use install_github() will find new features that may not be officially announced/ready.
• con: 🤕 cognitive load (rarely): during hotfixes the process is slightly more complex because a new branch must be created from a tag instead of main, tagged and released once approved while the PR is still open, and finally merged into main.
  • amelioration: The process for releasing hotfixes with a development branch is not that different in the number of steps: create a branch from main (against normal workflow), PR, merge and release, and then make PR from main into devel.

[elray1]

i will say, another con to "main identical to release (dev happens on branches)" that Anna mentioned today is that for testing/trying out functionality on branches that interact between different packages, things can get complicated. It can be done, but is certainly annoying.

[zkamvar]

Thank you for adding that! It was my original intention to add that to the list but then I got so far in the weeds 😅

[lshandross]

1 from me as well. I've often felt confused by having each merged pull request be a release, and setting up a more robust and formalized process makes more sense to me.

[zkamvar]

Closing this discussion because we now have guidelines in hubDevs: https://hubverse-org.github.io/hubDevs/articles/release-checklists.html