Initial version

Signed-off-by: Juan Bustamante <[email protected]>
buildpacks · Sep 14, 2023 · a75e8bd · a75e8bd
1 parent 75af1f3
commit a75e8bd
Showing 1 changed file with 320 additions and 0 deletions.
diff --git a/text/0000-multiarch-builders-and-package.md b/text/0000-multiarch-builders-and-package.md
@@ -0,0 +1,320 @@
+# Meta
+[meta]: #meta
+- Name: Multi-arch support for builders and buildpack packages
+- Start Date: 2023-09-14
+- Author(s): @jjbustamante
+- Status: Draft <!-- Acceptable values: Draft, Approved, On Hold, Superseded -->
+- RFC Pull Request: (leave blank)
+- CNB Pull Request: (leave blank)
+- CNB Issue: (leave blank)
+- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC)
+
+# Summary
+[summary]: #summary
+
+The problem for adding support for multi-arch buildpacks can be divided into three parts:
+1. Support buildpack authors to **migrate their existing buildpacks** to support multi-arch .
+2. Support buildpack authors to **create new buildpacks and builders** that handle multi-arch from the beginning.
+3. Support application developers to **create application images** for different os/arch 
+
+The purpose of this RFC is to add the capability to the commands:
+
+- `pack buildpack package`
+- `pack builder create`
+
+to output OCI artifacts for different OS and architectures and solves the second statement mentioned above. 
+
+# Definitions
+[definitions]: #definitions
+
+Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with.
+
+# Motivation
+[motivation]: #motivation
+
+- Why should we do this?
+
+The uses of ARM architecture in the cloud and edge computing has been growing rapidly. The CNCF community has been also growing in the last years, and there is a need to support multi-arch for all the projects. The buildpacks community is not an exception, issues like:
+- [It would be nice to support easily creating a manifest list packages and builders](https://github.com/buildpacks/pack/issues/1460)
+- [Provide a way to specify desired platform when creating packages and builders](https://github.com/buildpacks/pack/issues/1459)
+- [Multi arch image build support](https://github.com/buildpacks/pack/issues/1570)
+
+Or the conversations around this topic in our [slack channel](https://cloud-native.slack.com/archives/C032LNSMY0P), even the [talk at Kubecon NA 2022](https://www.youtube.com/watch?v=Sdr5axlOnDI&list=PLj6h78yzYM2O5aNpRM71NQyx3WUe1xpTn&index=76) demonstrate the interest from the community in this feature.
+
+- What use cases does it support?
+
+- What is the expected outcome?
+
+# What it is
+[what-it-is]: #what-it-is
+
+The end-user for this proposal are **Buildpack authors**, we expect to improve their user experience when creating multi-architecture buildpacks and builders as follows
+
+## Buildpack Package
+
+The following sections describe the functionality with some examples
+
+### Set up the buildpack scaffolding
+
+Let's use our buildpack author guide and create a new filesystem structure to handle a multi-arch buildpack.
+
+<!-- Specifies the platform in the form os/arch[/variant][:osversion] -->
+
+```bash
+pack buildpack new examples/ruby \
+--api 0.12 \
+--path ruby-buildpack \
+--version 0.0.1 \
+--platform "linux/amd64" --platform "linux/arm64" --platform "windows/amd64:10.0.20348.1970"
+```
+
+This command will create `ruby-buildpack` directory which contains buildpack.toml and folders for the different `os/arch[/variant]:[osversion] defined by the user and 
+bin/build, bin/detect files inside those folders.
+
+```bash
+ruby-buildpack
+├── buildpack.toml
+├── linux-amd64
+│         └── bin
+│             ├── build
+│             └── detect
+├── linux-arm64
+│         └── bin
+│             ├── build
+│             └── detect
+└── windows-amd64
+    └── 10.0.20348.1970
+        └── bin
+            ├── build.bat
+            └── detect.bat
+
+```
+
+The `buildpack.toml` created will be populated with a content similar to this one:
+
+```toml
+# Buildpack API version
+api = "0.12"
+
+# Buildpack ID and metadata
+[buildpack]
+  id = "examples/ruby"
+  version = "0.0.1"
+
+# List of targets operating systems, architectures and versions
+
+[[targets]]
+os = linux"
+arch = "amd64"
+
+[[targets.distributions]]
+name = ""
+
+[[targets]]
+os = linux"
+arch = "arm64"
+
+[[targets.distributions]]
+name = ""
+
+[[targets]]
+os = windows"
+arch = "amd64"
+
+[[targets.distributions]]
+name = ""
+versions = ["10.0.20348.1970"]
+
+```
+
+With this new folder structure, Buildpack authors are able to separate their binaries and dependencies for each os family and package them separately.
+
+### Package a buildpack
+
+Let's suppose a Buildpack author is ready to package their multi-arch buildpacks, following our [guide](https://buildpacks.io/docs/buildpack-author-guide/package-a-buildpack/) and keeping the previous structure, we will need to create a `package.toml` file and then invoke a `pack` command
+
+The `package.toml` file will be similar to the one describe in our documentation:
+
+```toml
+[buildpack]
+uri = "samples/buildpacks/hello-universe/"
+
+[[dependencies]]
+uri = "samples/buildpacks/hello-moon"
+
+[[dependencies]]
+uri = "docker://cnbs/sample-package:hello-world"
+```
+
+But in this case, we actually **remove** the [platform](https://buildpacks.io/docs/reference/config/package-config/) section because that information will be taken from the `buildpack.toml`.
+
+Packaging a multi-arch buildpack will require the output to be **publish** to a registry or **saved on disk** in OCI layout format because we can't save all the artifacts in the docker daemon.
+
+```bash
+pack buildpack package my-buildpack --config ./package.toml --publish
+```
+
+Or
+
+```bash
+pack buildpack package my-buildpack.cnb --config ./package.toml --format file
+```
+
+If any of these flags are set, the default behavior will be to determine the os/arch for the host machine and package the buildpack for that configuration only.
+
+## Buildpack Builder
+
+<!--
+This provides a high level overview of the feature.
+
+- Define any new terminology.
+- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor.
+- Explaining the feature largely in terms of examples.
+- If applicable, provide sample error messages, deprecation warnings, or migration guidance.
+- If applicable, describe the differences between teaching this to existing users and new users.
+-->
+
+# How it Works
+[how-it-works]: #how-it-works
+
+<!--
+This is the technical portion of the RFC, where you explain the design in sufficient detail.
+
+The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.
+-->
+
+## Buildpack Package
+
+### Current structure
+
+```toml
+[buildpack]
+uri = "<some URL or path>"
+
+[extension]
+uri = "<some URL or path>"
+
+[[dependencies]]
+uri = "<some URL or path>" 
+image = "<some image ref>"
+
+[platform]
+os = "[linux|windows]"
+```
+
+```bash
+buildpack
+├── bin
+│   ├── build
+│   └── detect
+└── buildpack.toml
+```
+
+### New structure
+
+#### package.toml
+
+```toml
+[buildpack]
+uri = "<some URL or path>"
+
+[extension]
+uri = "<some URL or path>"
+
+[[dependencies]]
+uri = "<some URL or path>" 
+image = "<some image ref>"
+
+```
+
+#### buildpack.toml
+
+Based on the [RFC-0096](https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md) we replaced the concept of `stacks` in `buildpack.toml` by a `target` which looks like:
+
+```toml
+[[targets]]
+os = "<operating system>"
+arch = "<system architecture>"
+variant = "<architecture variant>"
+[[targets.distributions]]
+name = "<distribution ID>"
+versions = ["<distribution version>"]
+```
+
+We can use this information to infer a folder structure like the following:
+
+```bash
+multi-arch-buildpack
+├── buildpack.toml
+└── {os}-{arch}               // optional
+    └── {variant}             // optional
+        └── {version}         // optional
+            └── bin
+                ├── build
+                └── detect
+```
+
+Preserving the following defaults:
+- If the `targets` list is empty and `/bin/build` is present
+    - A target with `os = "linux"` and `arch = "amd64"` is assumed by tools reading `buildpack.toml`.
+- If the `targets` list is empty and `/bin/build.bat` or `/bin/build.exe` is present
+    - A target with `os = "windows"` and `arch = "amd64"` is assumed by tools reading `buildpack.toml`.
+
+
+# Migration
+[migration]: #migration
+
+This section should document breaks to public API and breaks in compatibility due to this RFC's proposed changes. In addition, it should document the proposed steps that one would need to take to work through these changes. Care should be give to include all applicable personas, such as platform developers, buildpack developers, buildpack users and consumers of buildpack images.
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+Why should we *not* do this?
+
+# Alternatives
+[alternatives]: #alternatives
+
+- What other designs have been considered?
+- Why is this proposal the best?
+- What is the impact of not doing this?
+
+# Prior Art
+[prior-art]: #prior-art
+
+- This RFC is a continuation of the work started with the proposal to add commands to handle manifest list in pack, see the [RFC](https://github.com/buildpacks/rfcs/pull/283)
+
+Discuss prior art, both the good and bad.
+
+# Unresolved Questions
+[unresolved-questions]: #unresolved-questions
+
+- What parts of the design do you expect to be resolved before this gets merged?
+- What parts of the design do you expect to be resolved through implementation of the feature?
+- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?
+
+# Spec. Changes (OPTIONAL)
+[spec-changes]: #spec-changes
+Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here.
+Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc.
+This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here.
+
+# History
+[history]: #history
+
+<!--
+## Amended
+### Meta
+[meta-1]: #meta-1
+- Name: (fill in the amendment name: Variable Rename)
+- Start Date: (fill in today's date: YYYY-MM-DD)
+- Author(s): (Github usernames)
+- Amendment Pull Request: (leave blank)
+
+### Summary
+
+A brief description of the changes.
+
+### Motivation
+
+Why was this amendment necessary?
+--->