Release retrospective for 3.9.0 (Developers/Translators/Contributors/Users)

[ann0see]

Similar to 3.8.0, I believe it makes sense to start a post release discussion about Release 3.9.0 - especially concerning the release process, but also the time in between 3.8.2 and 3.9.0. I will try to follow the procedure by @hoffie from #1825.

Since it took quite some time to get the release shipped, and the huge list of features, there are probably some points we should discuss.

The following points are my personal impression:

The good:

• Even though life has returned mostly back to normal, the Jamulus community is still strong and active. People help out (thanks to all contributors discussing in issues, pull requests,...), propose and implement new features and help Jamulus getting constantly improved. Especially a lot of refactorings, translations and long due features (JSON-RPC API, ARM support, Qt6) have been implemented
• Our release checklist seems to be detailed and good enough, such that even people who haven't been active in specific stages of the release process can execute them correctly. Thanks to @pljones our release manager for this release! You did a great job (and finally moved it to the final release)
• Under the hood improvements such as Qt6 support, the Autobuild refactoring by @hoffie etc. help to make Jamulus future proof. A lot of cleanups are probably following (Sound API changes by @pgScorpio and a root folder cleanup by me).
• @ignotus666 and others are working on finally getting the website (and app translations) in sync and easier. Although the translating of the website has improved with the introduction of po4a, Weblate might bring a more translator friendly layout and thus more and easier translation.

The bad:

• Probably number one is the long release process and the time between releases. Jamulus is a free time project and we don't have a release pressure, however, we should at least release nightly builds much more often to find bugs in early development stages.
• Connected to number 1: Big releases also introduce a big changelog. While the Changelog script is very well written and allows us to save a lot of time, we might want to introduce a combine feature, to merge multiple related PRs together. Honestly, nobody wants to read a Changelog entry for every PR since we have a commit history. Also the goal of a Changelog should be user focused while the commit history should be developer focused.
• Automation e.g. of the conflicting keyboard shortcuts .pl script or a general release script or action might simplify the release process and remove some manual actions. For example, the translation issues could be opened automatically, as soon as a checkbox or comment on a release issue is given
• The website process is not documented well enough yet, but improved and documented by @gilgongo I think it's good to look over it from time to time, or - even better - include it into the release checklist
• We sometimes lack specialists for specific topics like the sound API, so PR's linger around for months. I'm not sure if corrados can help out for some specific PRs.
• Some translations are getting out of date on both the website and the app. We should decide when or if we drop them.

All in all, it's great we got the release finally out. Again huge thanks to all the contributors and @pljones for pushing it forward.

Happy to hear about impressions, ideas etc for the next release and the time between. No matter if you're a translator @jamulussoftware/translators Main-Dev @jamulussoftware/maindevelopers user or anything else.

[pljones]

I was pleasantly surprised by how "easy" most of the release process was to follow - thanks to everyone who has in the past worked on it. I guess I was a good guinea pig for test purposes! The hard part was the website process as I've absolutely zero knowledge in that area, nor the tooling. Whilst I don't know if the app process documentation could be used by someone without knowledge of building the app, I've not worked with releasing Jamulus before but felt it was reasonable.

I guess some of this was helped as I've been a full time project manager, too!

There were a few places where I moved things around in the template, where I split steps into simpler steps to keep track, where dependencies needed making clearer and so on. Hopefully those can be picked up where we see fit.

Tooling... well, automation can always be improved 😀 ... It was certainly good enough and much better than not having any.

I'd have to pick back through my comments on the Release issue to find everything of note - I tried to mention them when they came up as I've a dreadful memory.

[hoffie]

Thanks @pljones for making the release happen and thanks @ann0see for the retro.

Probably number one is the long release process and the time between releases. Jamulus is a free time project and we don't have a release pressure, however, we should at least release nightly builds much more often to find bugs in early development stages.

I have nothing against more nightlies, but I'm also questioning their usefulness or user exposure. They certainly do help shipping a certain state to users who want to make use of a specific improvement before a real release happens. But based on previous feedback requests, I'm questioning how many people actually do use those.

This release had a really high number of changes which are not user-visible (but which were certainly necessary). Those took time, those blew up the ChangeLog. For me, it was also some kind of a blocker for releasing 3.9.0 without any "real" features (at some time, JSON-RPC was the only one, I think; and while I do find it very useful and important to have, it's not the kind of feature which the average Jamulus user is going to understand; having at least some end-user visible improvement is helpful for explaining new versions; the accessibility improvements are a good sign, for example).

While the Changelog script is very well written and allows us to save a lot of time, we might want to introduce a combine feature, to merge multiple related PRs together. Honestly, nobody wants to read a Changelog entry for every PR since we have a commit history. Also the goal of a Changelog should be user focused while the commit history should be developer focused.

A combine feature would be helpful, but looking at the size and complexity of the script, I'm also a bit vary of adding too much. As it stands, the CHANGELOG still needs manual curation in order to be useful. I can't judge how much work the merging part currently is (given that we agree on the "use the exact same text as the related PR" part). I was planning to participate in the CHANGELOG curation stuff, but I didn't make it, so I think all this work had to be done by @pljones alone.

Automation e.g. of the conflicting keyboard shortcuts .pl script or a general release script or action might simplify the release process and remove some manual actions. For example, the translation issues could be opened automatically, as soon as a checkbox or comment on a release issue is given

Yes, there's certainly room for more improvement. Needs someone to do it and needs to be maintenable. :)

We sometimes lack specialists for specific topics like the sound API, so PR's linger around for months.

Yes, this is a hard problem which was very relevant, but hasn't been solved during the 3.9.0 cycle.

Some translations are getting out of date on both the website and the app. We should decide when or if we drop them.

For the app, my gut feeling is that we should keep translations even if larger parts start appearing untranslated. It might still help users and it might help gather new translators for those languages. Maybe adding some hint about that fact for languages below a given translation threshold would help.

[jujudusud]

I'm already happy to work on the translation and a little bit on the code of the app (even if i am actually learning how to code in C).
I think we have done a lot of good work, with a good method and reporting.
The deployement of the po4 stuff for the translation is a really good improvement and i wait the weblate step to see if it is even better.
Let us see what happens next!

[gilgongo]

I always feel it's hard to know whether the things tagged for a release are in a state where they can be merged in time to hit our dates. Things tend to get tagged for a release way before we even know when the release date will be. This seems a bit odd to me, and we usually have to do some de-tagging at some point. Is there a reason why we do that and not decide on a convenient date for a release, then discuss and tag the things we think we can include?

[pljones]

I'd worry somethings might never make it then.

[ann0see]

Exactly. I'd say the tagging is more of an overview at the beginning and becomes more binding as soon as we get to the end of a release.

[pljones]

Like the difference between 3.10.0 and 3.9.1 -- we know we want to get the 3.9.1 stuff out before 3.10.0... just not when precisely.

[ann0see]

I'll look at this once I have more time. In the meantime, we should find actionable points and update the release checklist if needed.