Documenting new infrastructure and migrating to it

[Olf0]

The new build and distribution infrastructure is now (2022-02-20) in place and in use:

1. At Sailfish-OBS and GitHub

1.1. New infrastructure

• The harbour-storeman sub-repository with the four packages harbour-storeman-installer, harbour-storeman-sfos3.2, harbour-storeman-sfos3.3 and harbour-storeman-sfos4.2.
• In the SailfishOS:Chum testing repository the four packages harbour-storeman-installer, harbour-storeman-sfos3.2, harbour-storeman-sfos3.3 and harbour-storeman-sfos4.2.
• In the regular SailfishOS:Chum repository the same four packages harbour-storeman-installer, harbour-storeman-sfos3.2, harbour-storeman-sfos3.3 and harbour-storeman-sfos4.2.

For CI at GitHub and Sailfish-OBS, plus building of release versions at GitHub, these additional build scripts and Sailfish-OBS repositories exits:

• Four new CI scripts build-devel.yml, build-sfos3.1.yml, build-sfos3.3.yml and build-sfos4.2.yml for immediate build testing of PRs. Their use is documented at the Storeman Wiki.
  I have to admit that I like the old build scripts with their higher degree of automatisation a lot, but then decided to keep things simple and easy to maintain.
  The last three of the aforementioned build scripts are also used to build release versions at GitHub, which are published at GitHub on the Storeman releases page.
• The harbour-storeman-ci sub-repository with the four packages harbour-storeman-installer, harbour-storeman-sfos3.2, harbour-storeman-sfos3.3 and harbour-storeman-sfos4.2. It is currently not really in use, because the webhooks at Sailfish-OBS are unusable due to a stale (since 2021-10-31) SSL (X.509) certificate. I triggered lbt to push for their renewal.
• The harbour-storeman-testing-ci sub-repository with the single package harbour-storeman-testing, which is fed from the master branch. Due to the non-functional webhooks it is currently also not really in use.

@mentaljam you have access as maintainer to all these repositories.

1.2. Migrating

Current Storeman installations use the mentaljam repository at SailfishOS-OBS for their self-updating. Hence we have to build a single new Storeman version there, with which the existing installations can update via the old infrastructure, and which ultimately uses the new infrastructure for self-updating.

@mentaljam, please carry out these steps to achieve that (I prepared it, so it is easy to Copy & Paste; GitHub also provides a JavaScript-based, overlaid "copy the whole code box"-button):

1.2.1

At https://build.sailfishos.org/project/meta/home:mentaljam please replace this XML-paragraph (lines 5 to 8)

  <repository name="sailfish_testing_armv7hl">
    <path project="sailfishos:latest" repository="latest_armv7hl"/>
    <arch>armv8el</arch>
  </repository>

with

  <repository name="4.3.0.15_i486">
    <path project="sailfishos:4.3.0.15" repository="latest_i486"/>
    <arch>i586</arch>
  </repository>
  <repository name="4.3.0.15_armv7hl">
    <path project="sailfishos:4.3.0.15" repository="latest_armv7hl"/>
    <arch>armv8el</arch>
  </repository>
  <repository name="4.3.0.15_aarch64">
    <path project="sailfishos:4.3.0.15" repository="latest_aarch64"/>
    <arch>aarch64</arch>
  </repository>
  <repository name="3.2.0.12_i486">
    <path project="sailfishos:3.2.0.12" repository="latest_i486"/>
    <arch>i586</arch>
  </repository>
  <repository name="3.2.0.12_armv7hl">
    <path project="sailfishos:3.2.0.12" repository="latest_armv7hl"/>
    <arch>armv8el</arch>
  </repository>

It is XML, the order does not matter!

1.2.2.

At https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-sfos3.2 please replace the whole (really all) meta-data there with

<package name="harbour-storeman-sfos3.2" project="home:mentaljam">
  <title>Storeman</title>
  <description>Storeman for Sailfish OS releases ≤ 3.2.1&#13;
&#13;
Mind that recent releases of Storeman require Sailfish OS ≥ 3.1.0.&#13;
Older Storeman releases are available at:&#13;
https://github.com/storeman-developers/harbour-storeman/releases&#13;
&#13;
Note that a "download on demand (DoD)" OBS-repository must be created by Jolla, before software can be built for a SailfishOS release at the SailfishOS-OBS.  It often takes some time after a new "general availability (GA)" SailfishOS release before the corresponding "DoD" repository is being made available, during which installing or updating Storeman by the Storeman Installer or Storeman's self-updating on a device with the new SailfishOS release already installed will fail (e.g., with "closed beta (cBeta)" and "early access (EA)" releases).  Hence one has to either manually set the last prior SailfishOS GA release in the SailfishOS:Chum GUI application or manually download and install or update Storeman, then.&#13;
&#13;
Source code repository:&#13;
https://github.com/storeman-developers/harbour-storeman&#13;
&#13;
</description>
  <build>
    <disable/>
    <enable repository="3.1.0.12_armv7hl"/>
    <enable repository="3.1.0.12_i486"/>
    <enable repository="3.2.0.12_armv7hl"/>
    <enable repository="3.2.0.12_i486"/>
    <enable repository="3.2.1.20_armv7hl"/>
    <enable repository="3.2.1.20_i486"/>
  </build>
</package>

1.2.3.

At https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-sfos3.3 please replace the whole (really all) meta-data there with

<package name="harbour-storeman-sfos3.3" project="home:mentaljam">
  <title>Storeman</title>
  <description>Storeman for Sailfish OS releases from 3.3.0 to 4.1.0&#13;
&#13;
Note that a "download on demand (DoD)" OBS-repository must be created by Jolla, before software can be built for a SailfishOS release at the SailfishOS-OBS.  It often takes some time after a new "general availability (GA)" SailfishOS release before the corresponding "DoD" repository is being made available, during which installing or updating Storeman by the Storeman Installer or Storeman's self-updating on a device with the new SailfishOS release already installed will fail (e.g., with "closed beta (cBeta)" and "early access (EA)" releases).  Hence one has to either manually set the last prior SailfishOS GA release in the SailfishOS:Chum GUI application or manually download and install or update Storeman, then.&#13;
&#13;
Source code repository:&#13;
https://github.com/storeman-developers/harbour-storeman&#13;
&#13;
</description>
  <build>
    <disable/>
    <enable repository="3.3.0.16_armv7hl"/>
    <enable repository="3.3.0.16_i486"/>
    <enable repository="3.4.0.24_armv7hl"/>
    <enable repository="3.4.0.24_i486"/>
    <enable repository="4.0.1.48_aarch64"/>
    <enable repository="4.0.1.48_armv7hl"/>
    <enable repository="4.0.1.48_i486"/>
    <enable repository="4.1.0.24_aarch64"/>
    <enable repository="4.1.0.24_armv7hl"/>
    <enable repository="4.1.0.24_i486"/>
  </build>
</package>

1.2.4.

At https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman please replace the whole (really all) meta-data there with

<package name="harbour-storeman" project="home:mentaljam">
  <title>Storeman</title>
  <description>Storeman for Sailfish OS releases ≥ 4.2.0&#13;
&#13;
Note that a "download on demand (DoD)" OBS-repository must be created by Jolla, before software can be built for a SailfishOS release at the SailfishOS-OBS.  It often takes some time after a new "general availability (GA)" SailfishOS release before the corresponding "DoD" repository is being made available, during which installing or updating Storeman by the Storeman Installer or Storeman's self-updating on a device with the new SailfishOS release already installed will fail (e.g., with "closed beta (cBeta)" and "early access (EA)" releases).  Hence one has to either manually set the last prior SailfishOS GA release in the SailfishOS:Chum GUI application or manually download and install or update Storeman, then.&#13;
&#13;
Source code repository:&#13;
https://github.com/storeman-developers/harbour-storeman&#13;
&#13;
</description>
  <build>
    <disable repository="3.1.0.12_armv7hl"/>
    <disable repository="3.1.0.12_i486"/>
    <disable repository="3.2.0.12_armv7hl"/>
    <disable repository="3.2.0.12_i486"/>
    <disable repository="3.2.1.20_armv7hl"/>
    <disable repository="3.2.1.20_i486"/>
    <disable repository="3.3.0.16_armv7hl"/>
    <disable repository="3.3.0.16_i486"/>
    <disable repository="3.4.0.24_armv7hl"/>
    <disable repository="3.4.0.24_i486"/>
    <disable repository="4.0.1.48_aarch64"/>
    <disable repository="4.0.1.48_armv7hl"/>
    <disable repository="4.0.1.48_i486"/>
    <disable repository="4.1.0.24_aarch64"/>
    <disable repository="4.1.0.24_armv7hl"/>
    <disable repository="4.1.0.24_i486"/>
  </build>
</package>

1.2.5.

At https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-installer please replace the whole (really all) meta-data there with

<package name="harbour-storeman-installer" project="home:mentaljam">
  <title>Storeman Installer</title>
  <description>Storeman installer is a "noarch" package which works on all CPU-architectures and should work on all Sailfish OS releases of the last couple of years.&#13;
&#13;
Mind that recent releases of Storeman proper require Sailfish OS ≥ 3.1.0.&#13;
Older Storeman releases are available at:&#13;
https://github.com/storeman-developers/harbour-storeman/releases&#13;
&#13;
Note that a "download on demand (DoD)" OBS-repository must be created by Jolla, before software can be built for a SailfishOS release at the SailfishOS-OBS.  It often takes some time after a new "general availability (GA)" SailfishOS release before the corresponding "DoD" repository is being made available, during which installing or updating Storeman by the Storeman Installer or Storeman's self-updating on a device with the new SailfishOS release already installed will fail (e.g., with "closed beta (cBeta)" and "early access (EA)" releases).  Hence one has to either manually set the last prior SailfishOS GA release in the SailfishOS:Chum GUI application or manually download and install or update Storeman, then.&#13;
&#13;
Source code repository:&#13;
https://github.com/storeman-developers/harbour-storeman-installer&#13;
&#13;
</description>
</package>

1.2.6.

You might delete the now superfluous package https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-testing

1.2.7.

At https://build.sailfishos.org/package/view_file/home:mentaljam/harbour-storeman-sfos3.2/_service?expand=1 please replace the whole (really complete) _service file there with

<services>
  <service name="tar_git">
    <param name="url">https://github.com/storeman-developers/harbour-storeman.git</param>
    <param name="branch">sfos3.2</param>
    <param name="revision"/>
    <param name="token"/>
    <param name="debian">N</param>
    <param name="dumb">N</param>
  </service>
</services>

1.2.8.

At https://build.sailfishos.org/package/view_file/home:mentaljam/harbour-storeman-sfos3.3/_service?expand=1 please replace the whole (really complete) _service file there with

<services>
  <service name="tar_git">
    <param name="url">https://github.com/storeman-developers/harbour-storeman.git</param>
    <param name="branch">sfos3.3</param>
    <param name="revision"/>
    <param name="token"/>
    <param name="debian">N</param>
    <param name="dumb">N</param>
  </service>
</services>

1.2.9.

At https://build.sailfishos.org/package/view_file/home:mentaljam/harbour-storeman/_service?expand=1 please replace the whole (really complete) _service file there with

<services>
  <service name="tar_git">
    <param name="url">https://github.com/storeman-developers/harbour-storeman.git</param>
    <param name="branch">sfos4.2</param>
    <param name="revision/">
    <param name="token"/>
    <param name="debian">N</param>
    <param name="dumb">N</param>
  </service>
</services>

1.2.10.

At https://build.sailfishos.org/package/view_file/home:mentaljam/harbour-storeman-installer/_service?expand=1 please replace the whole (really complete) _service file there with

<services>
  <service name="tar_git">
    <param name="url">https://github.com/storeman-developers/harbour-storeman-installer.git</param>
    <param name="branch">master</param>
    <param name="revision/">
    <param name="token"/>
    <param name="debian">N</param>
    <param name="dumb">N</param>
  </service>
</services>

2. At OpenRepos

Unfortunately no shared access is possible (without resorting to a shared secret, which is infeasible) at OpenRepos. My plan is to create a new Storeman Installer page under my account, which is based on the overhauled text at the front-page of Storeman Installer at GitHub.

But this is not there yet and not at all sufficient, because many other pages link to your Storeman Installer page at OpenRepos and the currently offered RPM there uses the old infrastructure (a vicious circle we are trying to break). Hence I am asking you to update the Storeman Installer RPM there for a last time with Storeman Installer v1.2.2, which is most easily to grab at GitHub, but you can also use the version build at the Storeman-OBS or SailfishOS:Chum testing (edit: Not a good idea, because it has the vendor set to chum).

Thanks & Cheers!

[mentaljam]

Thank you very much for such detailed instructions! I hope I've done everything right (if I'm wrong, I must be a complete idiot XD

[mentaljam]

BTW maybe it is possible to transfer a repository at OpenRepos? We should ask @custodian

[Olf0]

Thank you very much for such detailed instructions! I hope I've done everything right (if I'm wrong, I must be a complete idiot XD

Thank you: This is looking good.

BTW maybe it is possible to transfer a repository at OpenRepos? We should ask @custodian

Well, the "repository" would be https://openrepos.net/user/6416/programs. Actually I do not want that, even though I already asked myself what the future of Mashka and Recorder is (plus Beerware, though not I am not personally interested in it). They likely also need a Sailjail configuration in order to continue to work on SFOS ≥ 4.4.0. If you want to, you can transfer their source code repositories here at GitHub to storeman-developers.

So this would be about transferring this software page. Well, it worked fine for Patchmanager (twice, already) to create a new OpenReops page and this is less critical now that SailfishOS:Chum exists. So in my opinion it is fine to leave things as they are and I may create a new page, when I have time for that.

Also note that ultimately it is just you and me here at storeman-developers and I will not do any serious C++ or QML coding (although I like to read your C++ code: It is well structured and looking good): Out of the four people mentioned in Jolla's "community news" only three really volunteered IMO (I believe that Sefriol only uttered his opinion, see [1] and [2], because he stated that "Creating own organisation does not make sense to me. I think putting it under chum would have been fine.", so I did not invite him), pherjung did not accept his invitation to storeman-developers (which is still open), but contributed once with testing (Calling out to test at FSO was quite successful, I will do this again in a better planned manner to have a pre-release tested) and poetaster (who was the only one to accept the invitation to storeman-developers besides you) primarily contributed with endless discussions (or here etc.), while his very few "code" contributions needed quite some quality assurance.

I hope it is O.K. for you to continue to be the main author and code maintainer of all software at storeman-developers, while I can take care of packaging, shell scripts, infrastructure, the publishing process, content of the issue tracker etc.

[poetaster]

Just a note, Olf. I maintain 16 apps which is admittedly, too much. My time is limited.

As for endless discussions, I'm done with that. But that also means I will limit my involvement since I'm uninterested in pointless discussions.

As for quality assurance, if anyone helped me, I believe I would have more time for QA.

[Olf0]

@poetaster, with a little temporal distance I see two principal aspects here:

1. The last four weeks have been exhausting for me, because I devoted almost all of my spare time to overhauling Storeman's infrastructure and getting Storeman and Storeman Installer fit for SFOS 4.4.0. Which turned out to be just in time, because SFOS 4.4.0 was made available for general consumption yesterday.

   Because you originally volunteered at FSO for maintaining Storeman and I merely became involved due to asking some "checks & balances" questions on the phone (primarily, if that is really feasible for you) my original understanding of the deal was very different: I support you with some organisational help. Thus my expectation was to do only that.

   But ultimately these were just my expectations, as neither you or me has provided an explicit commitment. As this work is done voluntarily, no one has formal obligations. Hence I have to live with my disappointment, that I ended up doing everything (except for replicating mentaljam's OBS configuration, which needed a lot of guidance from my side to resolve misconceptions).

   Thus I am sorry that some of my statements may sound like accusations, as I did not intend to hide being disappointed and intended to dryly denote the reasons why.

2. Just a note, Olf. I maintain 16 apps which is admittedly, too much. My time is limited.

   This is exactly what I addressed weeks ago on the phone with you. You replied that this is well feasible for you, so I trusted your assessment.

   As for endless discussions, I'm done with that. But that also means I will limit my involvement since I'm uninterested in pointless discussions.

   Bluntly stated, I cannot perceive any loss at this point.

   As for quality assurance, if anyone helped me, I believe I would have more time for QA.

   Sorry, from my perspective this is an awkward way to bail out: You volunteered publicly to maintain Storeman, which I ended up doing, so I asked you to perform at least some quality assurance and the reply is, "maybe, if you devote time to quality assurance of my other projects"!?!

[Olf0]

@mentaljam, finally I understood by analysing the tar_git shell script, how to handle the version numbers at Sailfish-OBS well, which in turn required to adapt the tag naming scheme here at GitHub. Additionally I fixed a packaging bug I introduced, which affected older SFOS versions and prevented self-updating Storeman for them. I expect this to be the last time I am asking you to update your OBS-repository.

Hence please accept the four OBS-requests I posed. Also please delete your extension-less package, to have consistent names in all OBS-repositories (yours, mine and SailfishOS:Chum). If I ever submit my packages to your repo in order to update your packages again, this will ease the process then, but generally clicking on "Trigger services" for all packages should be sufficient to update to the latest release version of Storeman / Storeman Installer with these overhauled _service files.

Optionally you might also update the package descriptions, which have been reworked (I often need some time to settle things), because OBS does not copy them as part of a package submission. This is most easily done by Copy&Paste of the <description> section from and to the metadata tab:

1. https://build.sailfishos.org/package/meta/home:olf:harbour-storeman/harbour-storeman-installer
   → https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-installer

2. https://build.sailfishos.org/package/meta/home:olf:harbour-storeman/harbour-storeman-sfos3.2
   → https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-sfos3.2

3. https://build.sailfishos.org/package/meta/home:olf:harbour-storeman/harbour-storeman-sfos3.3
   → https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-sfos3.3

4. The description of harbour-storeman-sfos4.2 should be copied when initially accepted.
   If this is not the case, then please also copy the <description> section:
   https://build.sailfishos.org/package/meta/home:olf:harbour-storeman/harbour-storeman-sfos4.2
   → https://build.sailfishos.org/package/meta/home:mentaljam/harbour-storeman-sfos4.2

   Please also check that the <build> section has been transferred correctly (I do not remember if OBS does that or if it also has to be copied manually) for the harbour-storeman-sfos4.2 package.

Thank you!

[poetaster]

@poetaster, with a little temporal distance I see two principal aspects here:

1. The last four weeks have been exhausting for me, because I devoted almost all of my spare time to overhauling Storeman's infrastructure and getting Storeman and Storeman Installer fit for SFOS 4.4.0.  Which turned out to be just in time, because SFOS 4.4.0 was made available for general consumption yesterday.
   Because [you originally volunteered at FSO for maintaining Storeman](https://forum.sailfishos.org/t/future-of-storeman/10545/12?u=olf) and I merely became involved due to asking some "checks & balances" questions on the phone (primarily, if that is really feasible for you) my original understanding of the deal was very different: I support you with some organisational help.  Thus my expectation was to do only that.
   But ultimately these were just my expectations, as neither you or me has provided an explicit commitment.  As this work is done voluntarily, no one has formal obligations.  Hence I have to live with my disappointment, that I ended up doing everything (except for replicating mentaljam's OBS configuration, which needed a [lot of guidance](https://github.com/storeman-developers/harbour-storeman/issues/163#issuecomment-1060741364) from my side to resolve misconceptions).
   Thus I am sorry that some of my statements may sound like accusations, as I did not intend to hide being disappointed and intended to dryly denote the reasons why.

2. > Just a note, Olf. I maintain 16 apps which is admittedly, too much. My time is limited.
   
   
   This is exactly what I addressed weeks ago on the phone with you.  You replied that this is well feasible for you, so I trusted your assessment.
   > As for endless discussions, I'm done with that. But that also means I will limit my involvement since I'm uninterested in pointless discussions.
   
   
   Bluntly stated, I cannot perceive any loss at this point.
   > As for quality assurance, if anyone helped me, I believe I would have more time for QA.
   
   
   Sorry, from my perspective this is an awkward way to bail out: You volunteered publicly to maintain Storeman, which I ended up doing, so I asked you to perform at least some quality assurance and the reply is, "maybe, if you devote time to quality assurance of my other projects"!?!

It was quickly clear that my input is not welcome. I would have simplified this mess to a single repo, removed the sharing feature and made certain that the breaking changes within the 3 branch were removed.

But you know better. And then proceeded to waste numerous peoples time. Which is why I have now also left this project. Good luck.

[Olf0]

@mentaljam,

[…] I already asked myself what the future of Mashka and Recorder is (plus Beerware, though not I am not personally interested in it). They likely also need a Sailjail configuration in order to continue to work on SFOS ≥ 4.4.0. If you want to, you can transfer their source code repositories here at GitHub to storeman-developers.

BTW, Mashka already has been reported to run on SFOS 4.4.0, but to do very little, likely due to Sailjail.

[Olf0]

Hi @mentaljam, please click four times on 🔍 → Accept request: https://build.sailfishos.org/project/requests/home:mentaljam

For details, see #266 (comment)

[mentaljam]

Hence please accept the four OBS-requests I posed.

Done! Thank you again for detailed instructions! I've also added you as a maintainer to these OBS packages

[Olf0]

I've also added you as a maintainer to these OBS packages

Thank you @mentaljam, that was very helpful, as it allowed me to update the packages, which are downloaded and installed by Storeman Installer < 1.2.0 and Storeman 0.2.9, 0.2.10, 0.2.11 from the SFOS-OBS.
As the OBS-repository migration has been successful (via Storeman Installer ≥ 1.2.0 and Storeman ≥ 0.2.12), I dropped my maintainer access to your SFOS-OBS-repository after building Storeman Installer 1.4.3 and Storeman 0.3.2 there (because I want to reduce the administrative work for me after each release; "3½" OBS repos are too many: yours, mine, SailfishOS:Chum-testing and the half being SailfishOS:Chum).

Now the final step left to complete the repository migration is to upload a Storeman Installer ≥ 1.3.1 (with v1.4.3 strongly suggested) to your "mentaljam" OpenRepos repository, see details and reasons there.