Add ADR for branching strategy

[rancher-max]

Proposed Changes

This PR adds an ADR to propose changing the way we currently use branches here on Github, ultimately changing the release process to allow for more flexibility.

Types of Changes

Verification

Linked Issues

Further Comments

[brandond]

I like this idea in that it lets us land things during code freeze.

My primary concern about this is how it will affect tracking things on GH. Picking on 1.24 for example - if we PR all our backports into a dev-v1.24, and then merge dev-v1.24 into release-1.24 or simply fast-forward release-1.24 up to the head of dev-v1.24 after code freeze, GH won't show that. If we look at the backport PR, it will only show the merge into the dev branch - it'll break the link between the commit and the merged PR on the branch we use to cut our actual releases.

Switching back and forth trying to cross-link commits across branches is already challenging when trying to follow a commit to master back into the various release branches and I think this would make that worse.

[matttrach]

When would the dev branches merge into the release branches?
I have a philosophical concern that this gets us further away from Continuous Integration, but I don't have any practical concerns to point out since we will just be continuously integrating into dev branches rather than release branches.

From a practical perspective, this reminds me of how KDM works, which causes a lot of pain in the release process.

[matttrach]

Is this to avoid additional tagging? The code freeze would still exist on the release branches, but not the dev branches, right? So in the case of a critical CVE which requires an additional release, the release team should cherry-pick from the dev branch to get the fix. Then, after release, we rebase to fast forward the release branch to the dev branch? It kinda seems like we could achieve the same results just using tags, right? (it would rearrange history, but so does the branch solution)

[matttrach]

It seems like what you want to get closer to is trunk development with release trains, but we need to isolate changes using a feature flagging system to make that possible.
One simple way to implement this kind of isolation without rearranging our entire system is to commit commented out code. Then create a PR to uncomment the code after release. There are ways to make this easier (open/close principle is one), and other ways to implement the isolation, but this is the easiest that comes to mind.

[rancher-max]

All the concerns noted here are very valid and are probably worth not accepting this, but I want to address them for the sake of communication.

@brandond

My primary concern about this is how it will affect tracking things on GH... Switching back and forth trying to cross-link commits across branches is already challenging when trying to follow a commit to master back into the various release branches and I think this would make that worse.

I agree. Changing the branching strategy here may necessitate a different backport strategy as well that makes cross-link irrelevant, but I don't know what that would look like or if it would make sense at all.

@matttrach

When would the dev branches merge into the release branches?

I think it should be as soon as "code freeze" would normally begin. The verbiage could be changed, but it would effectively operate the same as it is today where no "new features" are to find their way into the release branches, but bug changes found from testing could. To go back to Brad's point, this could make tracking very difficult because during this period, there could be a change needed on like release-v1.25, which then would need to be brought to dev-v1.25 as well (where potentially other new features have already been added since that branch is never frozen with this process).

Is this to avoid additional tagging? ... It kinda seems like we could achieve the same results just using tags, right? (it would rearrange history, but so does the branch solution)

The way I envision that is that the CVE changes would be made to the release branches themselves, then ported over to the dev branches. Of course this has its series of own complications. It might be possible to achieve the same results just using tags, yes!

It seems like what you want to get closer to is trunk development with release trains

It's hard to answer this... we're currently basically doing trunk-based dev. This just provides a way to continuously merge code and develop new features while allowing for a single stable trunk that can be released from for critical changes.

---

All in all, the overarching goal is to get to a place where:

1. Releasing an r2 for critical CVEs or bugfixes can happen quickly and be isolated to ONLY those changes
2. There never has to be a code-freeze that slows down development.

There is likely a better way to achieve those goals than what this ADR presents.

[brandond]

As per slack discussion - pushing the new release branch should be the responsibility of the engineer that merges the PR bumping Kubernetes to the new minor. The new branch should diverge at the first commit using the new Kubernetes minor. Once the branch is ready for release, it can be handed off to the release captain.

Suggested change

	- Minor release captain will have to cut the new branch as soon as they bring in that new minor version.
	- Minor release captain will have to cut the new branch as soon as they bring in that new minor version.

[brandond]

yay codespell:

./developer-docs/updating_rke2_charts.md:22: consumin ==> consuming
Error: Process completed with exit code 6

[rancher-max]

yay codespell:

./developer-docs/updating_rke2_charts.md:22: consumin ==> consuming
Error: Process completed with exit code 6

huh, I wonder how that got in in the first place since it's not touched here as part of this PR!

[brandond]

codespell is externally updated and will randomly pick up new words. It doesn't have a full dictionary, just whatever stuff the maintainers add. It's quite irritating.

I fixed the issue in another already-merged PR, if you want to rebase.