What does plain+v0 mean?

[fgiloux]

From the documentation

the `plain+v0` bundle format is at schema version v0, which means it's an experimental format that is subject

I would be interested in the rational for the naming. It does not seem to be a mime type, although the use of '+' is one of its specificity. More importantly v0 does not convey much information in terms of the stability of the format, besides it is experimental. It is also not clear how the version number may evolve. Is v1 the only possible next version? Don't you see any benefits in differentiating between minor and major changes?

I think that there would be value in following Kubernetes API versioning scheme. API evolution commitments have already been defined and are known by the Kubernetes community. Alternatively a similar document would need to be created or referenced.

[joelanford]

This follows the same pattern as was defined for CSV-based bundles, as is defined here: https://olm.operatorframework.io/docs/tasks/creating-operator-bundle/

Specifically, the CSV-based bundle has the following annotation/image label:

annotations:
  operators.operatorframework.io.bundle.mediatype.v1: "registry+v1"

However, there was very little discussion around the choice of following the pattern and using "plain+v0", and I absolutely think there should be some discussion on the entire subject of bundle metadata.

Some very rough thoughts here: #172

[fgiloux]

Right, it seems reasonable to do things in this order:

1. discuss the content of a bundle, including metadata, current and foreseen evolution
2. discuss the format of a bundle, current and foreseen evolution
3. discuss versioning strategies

I looked at the links you shared but did not see much content that would go into metadata. Here is what I can imagine on top of my head:

• cluster constraints like:
  • Kubernetes version compatibility
  • plain provisioner version compatibility
  • server architecture, e.g. AMD, ARM compatibility
• dependencies on other bundles or APIs
• description
• image
• compliance aspects like FIPS encryption

Is this issue the right place for starting this discussion or would you prefer a different media?

[joelanford]

Let's start an actual GitHub discussion, maybe? I'll convert this issue to a discussion.

[joelanford]

I have so many thoughts here:

Immutability

I think the bundle itself should contain only immutable metadata that is inherent to the bundle itself. If some one wants to delete or change something about a bundle's metadata, they need to build and release a new bundle.

Making retroactive assertions about existing bundles (e.g. "this bundle is deprecated") should be handled via some other means external to the bundle itself.

Versioning

Partially motivated by the immutability topic (and partially by the need to rebuild bundles to fix CVEs in base images), I think it is important that we draw a distinction between:

• the version of the bundled software (upstream project version of the operator, operand versioning is separate)
• the revision of the bundle itself (downstream release version of the operator, operand versioning is separate)

This is essentially saying that we should have something very similar to RPM's version and release fields. And maybe epoch as well? https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/

Required vs. Optional

Should bundle metadata be required? IMO, no. At least not in the context of a RukPak provisioner. We clearly have an example already of a plain provisioner and bundle format that require no metadata. We need to consider the consumer(s) of the bundle and be careful to avoid placing overly burdensome requirements in places where they aren't actually necessary.

If I'm someone who doesn't care about resolution, upgrades, or building catalogs, why should I be forced to think about metadata related to those things?

Another example: Rukpak currently has an alpha-level helm chart provisioner. Today, we can take Helm charts right off the shelf and install them. If we add a requirement that charts include extra metadata that isn't already present in the chart metadata, users can't use any of the existing helm charts out there.

[fgiloux]

I do think that there is value in defining the purpose of the metadata before discussing how it should get handled. Should I assume that you agree with what I mentioned and that you don't see other purpose?

I am getting really confused with how you qualify a bundle, i.e. rebuilding bundles. I was instructed that a bundle is a directory with a bunch of manifests. Can we agree on it or can you let me know what you use the term for?

Immutability

I agree that immutability is very important, not only for metadata but for the resource manifests as well. Container images are by nature immutable, a git commit is too. This is however not the case of a git branch and although a git tag is theoretically immutable its identity is not protected so that it is possible to delete and recreate the tag pointing to a different commit. ConfigMaps can be made immutable but first they only exist on a cluster, secondly their identity is not protected either (delete/recreate). In my mind it means that only OCI Images and git commits are suitable for production or shipping to third parties. Other mechanisms may be "convenient" for development. If we agree on that this should be clearly documented, also at the bundle resource API level.

Making retroactive assertions about existing bundles (e.g. "this bundle is deprecated") should be handled via some other means external to the bundle itself.

This could be captured at the catalog level.

Versioning

the version of the bundled software (upstream project version of the operator, operand versioning is separate)
the revision of the bundle itself (downstream release version of the operator, operand versioning is separate)

This is missing my point. By versioning I really meant plain+v0, equivalent to RPM v4.19. This is not related to what users ship but the tooling we (or third parties) provide. We need to capture that the bundle is compatible with a plain provisioner version. We may make changes to the format that are backward compatibles (e.g. non mandatory field addition in the metadata) or not (e.g. change in the filesystem structure)

Regarding your two points I am fine with them. The generation of the release number should however be automated in 99% of the cases.

Required vs. Optional

Should bundle metadata be required? IMO, no.

We should use sane defaults every time we can (conventions over configuration). It would be good to make them clear.
I am not sure to follow your helm charts example. As long as the metadata does not need to be embedded into the charts manifests it would be possible to reuse existing charts and still configure additional metadata, wouldn't it?