From 4f73fe4721644103b53e098a68b2bc60240c0bf1 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 30 Jun 2021 17:43:21 -0400 Subject: [PATCH 001/372] Add RFC: Support Dockerfiles Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 113 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 text/0000-dockerfiles.md diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md new file mode 100644 index 000000000..ebe9cf409 --- /dev/null +++ b/text/0000-dockerfiles.md @@ -0,0 +1,113 @@ +# Meta +[meta]: #meta +- Name: Support Dockerfiles +- Start Date: 2021-06-30 +- Author(s): sclevine +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: [RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md), [RFC#167](https://github.com/buildpacks/rfcs/pull/167) +- Depends on: [RFC#172](https://github.com/buildpacks/rfcs/pull/172) + +# Summary +[summary]: #summary + +This RFC introduces functionality for customizing base images, as an alternative to stackpacks. + +# Motivation +[motivation]: #motivation + +Relying on Dockerfiles for base image generation and manipulation allows us to apply buildpacks only to the problem that they solve best: managing application runtimes and dependencies. + +# What it is +[what-it-is]: #what-it-is + +This RFC proposes that we replace stackpacks with multi-purpose build-time and runtime Dockerfiles. + +# How it Works +[how-it-works]: #how-it-works + +Note: kaniko, BuildKit, and/or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. + +### App-specified Dockerfiles + +A buildpack app may have a build.Dockerfile and/or run.Dockerfile in its app directory. A run.Dockerfile is applied to the selected runtime base image after the detection phase. A build.Dockerfile is applied to the build-time base image before the detection phase. + +Both Dockerfiles must accept `base_image` and `build_id` args. The `base_image` arg allows the lifecycle to specify the original base image. The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. + +A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of app-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before `run.Dockerfile` is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. + +### Platform-specified Dockerfiles + +The same Dockerfiles may be used to create new stacks or modify existing stacks outside of the app build process. For both app-specified and stack-modifying Dockerfiles, any specified labels override existing values. + +Dockerfiles that are used to create a stack must create a `/cnb/stack/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after any run.Dockerfile or build.Dockerfile is applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. + +### Examples + +run.Dockerfile used to create a runtime base image: + +``` +ARG base_image +FROM ${base_image} +ARG build_id=0 + +LABEL io.buildpacks.image.distro=ubuntu +LABEL io.buildpacks.image.version=18.04 +LABEL io.buildpacks.rebasable=true + +ENV CNB_USER_ID=1234 +ENV CNB_GROUP_ID=1235 + +RUN groupadd cnb --gid ${CNB_GROUP_ID} && \ + useradd --uid ${CNB_USER_ID} --gid ${CNB_GROUP_ID} -m -s /bin/bash cnb + +USER ${CNB_USER_ID}:${CNB_GROUP_ID} + +COPY genpkgs /cnb/stack/genpkgs +``` + +run.Dockerfile present in an app directory that always installs the latest version of curl: +``` +ARG base_image +FROM ${base_image} +ARG build_id=0 + +LABEL io.buildpacks.rebasable=true + +RUN echo ${build_id} + +RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* +``` + +Unsafe run.Dockerfile present in an app directory: +``` +ARG base_image +FROM ${base_image} +ARG build_id=0 + +LABEL io.buildpacks.rebasable=false + +RUN curl -L https://example.com/mypkg-install | sh # installs to /opt +``` + +# Drawbacks +[drawbacks]: #drawbacks + +- Involves breaking changes. +- Buildpacks cannot install OS packages directly, only select runtime base images. + +# Alternatives +[alternatives]: #alternatives + +- Stackpacks + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- Should `genpkgs` be part of this proposal? Opinion: Yes, otherwise it's difficult to maintain a valid SBoM. + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +This RFC requires extensive changes to all specifications. From ef2b23012fc53ed0eb1f037250167655520ae53a Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 1 Jul 2021 00:02:35 -0400 Subject: [PATCH 002/372] RFC: Support Dockerfiles: remove stack references Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index ebe9cf409..d775abc40 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -27,7 +27,7 @@ This RFC proposes that we replace stackpacks with multi-purpose build-time and r # How it Works [how-it-works]: #how-it-works -Note: kaniko, BuildKit, and/or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. +Note: kaniko, buildah, BuildKit, the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. ### App-specified Dockerfiles @@ -39,9 +39,9 @@ A runtime base image may indicate that it preserves ABI compatibility by adding ### Platform-specified Dockerfiles -The same Dockerfiles may be used to create new stacks or modify existing stacks outside of the app build process. For both app-specified and stack-modifying Dockerfiles, any specified labels override existing values. +The same Dockerfiles may be used to create new base images or modify existing base images outside of the app build process. For both app-specified and base-image-modifying Dockerfiles, any specified labels override existing values. -Dockerfiles that are used to create a stack must create a `/cnb/stack/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after any run.Dockerfile or build.Dockerfile is applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. +Dockerfiles that are used to create a base image must create a `/cnb/image/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after any run.Dockerfile or build.Dockerfile is applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. ### Examples @@ -64,7 +64,7 @@ RUN groupadd cnb --gid ${CNB_GROUP_ID} && \ USER ${CNB_USER_ID}:${CNB_GROUP_ID} -COPY genpkgs /cnb/stack/genpkgs +COPY genpkgs /cnb/image/genpkgs ``` run.Dockerfile present in an app directory that always installs the latest version of curl: From e21b2af2ddf2a8864626d9e1e2a6a88fce209524 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 14 Jul 2021 23:38:21 -0400 Subject: [PATCH 003/372] RFC: Support Dockerfiles: add builder-specified Dockerfiles Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 68 +++++++++++++++++++++++++++++++--------- 1 file changed, 54 insertions(+), 14 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index d775abc40..fa231c167 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -27,25 +27,65 @@ This RFC proposes that we replace stackpacks with multi-purpose build-time and r # How it Works [how-it-works]: #how-it-works -Note: kaniko, buildah, BuildKit, the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. +Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. -### App-specified Dockerfiles +### Builder-specified Dockerfiles -A buildpack app may have a build.Dockerfile and/or run.Dockerfile in its app directory. A run.Dockerfile is applied to the selected runtime base image after the detection phase. A build.Dockerfile is applied to the build-time base image before the detection phase. +A builder may specify any number of executable "hooks" in `/cnb/hooks.d/`. +Hook files in this directory are executed in the context of app directory and must output Dockerfiles that are applied to the build-time and runtime base images before an image is built. -Both Dockerfiles must accept `base_image` and `build_id` args. The `base_image` arg allows the lifecycle to specify the original base image. The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. +If a hook exits with a non-zero status value, the build fails. If a hook exits with a zero status value and no output, the hook is ignored. Directories and non-executable files are ignored. + +Each executable must be in the format `/cnb/hooks.d/.(build.|run.|)`, where build, run, or empty specify the build-time base image, runtime base image, or both bash images, respectively. The only valid format is `Dockerfile`, although support for, e.g. LLB JSON, could be added in the future. + +A runtime Dockerfile is applied to the selected runtime base image after the detection phase. +A build-time Dockerfile is applied to the build-time base image before the detection phase. + +Both Dockerfiles must accept `base_image` and `build_id` args. +The `base_image` arg allows the lifecycle to specify the original base image. +The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of app-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before `run.Dockerfile` is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. + +#### Example: App-specified Dockerfile Hook + +This example hook would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." +The app developer can decide whether the extensions are rebasable. + +##### `/cnb/hooks.d/app.build.Dockerfile` +``` +#!/bin/sh +cat build.Dockerfile +``` +##### `/cnb/hooks.d/app.run.Dockerfile` +``` +#!/bin/sh +cat run.Dockerfile +``` + +#### Example: RPM Dockerfile Hook + +This example hook would allow a builder to install RPMs for each language runtime. + +Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. + +##### `/cnb/hooks.d/app.Dockerfile` +``` +#!/bin/sh +[[ -f Gemfile.lock ]] && cat /cnb/hooks.d/app.Dockerfile.d/Dockerfile-ruby +[[ -f package.json ]] && cat /cnb/hooks.d/app.Dockerfile.d/Dockerfile-node +``` + ### Platform-specified Dockerfiles -The same Dockerfiles may be used to create new base images or modify existing base images outside of the app build process. For both app-specified and base-image-modifying Dockerfiles, any specified labels override existing values. +The same Dockerfile format may be used to create new base images or modify existing base images outside of the app build process. Any specified labels override existing values. -Dockerfiles that are used to create a base image must create a `/cnb/image/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after any run.Dockerfile or build.Dockerfile is applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. +Dockerfiles that are used to create a base image must create a `/cnb/image/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after all Dockerfiles are applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. -### Examples +### Example Dockerfiles -run.Dockerfile used to create a runtime base image: +Dockerfile used to create a runtime base image: ``` ARG base_image @@ -67,35 +107,35 @@ USER ${CNB_USER_ID}:${CNB_GROUP_ID} COPY genpkgs /cnb/image/genpkgs ``` -run.Dockerfile present in an app directory that always installs the latest version of curl: +run.Dockerfile for use with the example `app.Dockerfile` hook that always installs the latest version of curl: ``` ARG base_image FROM ${base_image} ARG build_id=0 -LABEL io.buildpacks.rebasable=true - RUN echo ${build_id} RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* ``` +(note: this Dockerfile disables rebasing as OS package installation is not rebasable) -Unsafe run.Dockerfile present in an app directory: +run.Dockerfile for use with the example `app.Dockerfile` hook that installs a special package to /opt: ``` ARG base_image FROM ${base_image} ARG build_id=0 -LABEL io.buildpacks.rebasable=false +LABEL io.buildpacks.rebasable=true RUN curl -L https://example.com/mypkg-install | sh # installs to /opt ``` +(note: rebasing is explicitly allowed because only a single directory in /opt is created) + # Drawbacks [drawbacks]: #drawbacks - Involves breaking changes. -- Buildpacks cannot install OS packages directly, only select runtime base images. # Alternatives [alternatives]: #alternatives From a1810ca08075da4c010df10679020f5d4131d8b4 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 15 Jul 2021 15:45:52 -0400 Subject: [PATCH 004/372] RFC: Support Dockerfiles: Fix reference to app-specified Dockerfiles Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index fa231c167..1ab34a336 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -45,7 +45,7 @@ Both Dockerfiles must accept `base_image` and `build_id` args. The `base_image` arg allows the lifecycle to specify the original base image. The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). -A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of app-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before `run.Dockerfile` is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. +A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before `run.Dockerfile` is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. #### Example: App-specified Dockerfile Hook From 82cbe67922d0fe3650cfddaa721ef3c8581ff620 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 15 Jul 2021 15:50:34 -0400 Subject: [PATCH 005/372] RFC: Support Dockerfiles: Remove run.Dockerfile reference Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 1ab34a336..7ae7b248b 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -45,7 +45,7 @@ Both Dockerfiles must accept `base_image` and `build_id` args. The `base_image` arg allows the lifecycle to specify the original base image. The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). -A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before `run.Dockerfile` is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. +A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. #### Example: App-specified Dockerfile Hook From a9daed9cf8da3bf318114b429a51fcc33dac608a Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Sun, 18 Jul 2021 16:58:43 -0400 Subject: [PATCH 006/372] RFC: Support Dockerfiles: address more feedback Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 41 ++++++++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 7ae7b248b..0e10f0e5c 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -31,15 +31,32 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap ### Builder-specified Dockerfiles -A builder may specify any number of executable "hooks" in `/cnb/hooks.d/`. -Hook files in this directory are executed in the context of app directory and must output Dockerfiles that are applied to the build-time and runtime base images before an image is built. +A builder image may include any number of "hook" files in `/cnb/hooks.d/`. +The `pack create-builder` command is used to copy hooks into the builder image via `builder.toml`. E.g., +```toml +[[hooks]] +name = "app-hook" +path = "./myhook" +``` +(Note: this format should not be considered final. Additional fields will be required to mark the format, target, etc. of the hook. These may be defined in subsequent spec PRs / sub-team RFCs.) + +Each hook file path must be in the format `/cnb/hooks.d/.(build.|run.|)(.out|)`, where: -If a hook exits with a non-zero status value, the build fails. If a hook exits with a zero status value and no output, the hook is ignored. Directories and non-executable files are ignored. +1. `build`, `run`, or empty specify the build-time base image, runtime base image, or both bash images, respectively. +2. The only valid format is `Dockerfile`, although support for, e.g. LLB JSON, could be added in the future. +3. If the `.out` suffix is present and the file is executable, the hook is executed in the context of the app directory, and its output must match ``. +4. If the `.out` suffix is not present, the contents of the file must match ``. -Each executable must be in the format `/cnb/hooks.d/.(build.|run.|)`, where build, run, or empty specify the build-time base image, runtime base image, or both bash images, respectively. The only valid format is `Dockerfile`, although support for, e.g. LLB JSON, could be added in the future. +Hook files are evaluated (either read or executed) and the resulting Dockerfiles are applied to the build-time and runtime base images in lexographical order by ``. +Executable hook files must not modify the app directory when executed and may be executed in parallel. +However, the resulting Dockerfiles must not be applied in parallel. +A Dockerfile intended for the runtime base image is applied after the detection phase. +A Dockerfile intended for the build-time base image is applied before the detection phase. -A runtime Dockerfile is applied to the selected runtime base image after the detection phase. -A build-time Dockerfile is applied to the build-time base image before the detection phase. +If an executable hook exits with a non-zero status value, the build fails. +If a executable hook exits with a zero status value and no output, the hook is ignored. +Directories are ignored. +Files at the top-level of `/cnb/hooks.d/` that do not match the specified file name format result in a build failure. Both Dockerfiles must accept `base_image` and `build_id` args. The `base_image` arg allows the lifecycle to specify the original base image. @@ -53,12 +70,12 @@ A runtime base image may indicate that it preserves ABI compatibility by adding This example hook would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." The app developer can decide whether the extensions are rebasable. -##### `/cnb/hooks.d/app.build.Dockerfile` +##### `/cnb/hooks.d/app.build.Dockerfile.out` ``` #!/bin/sh cat build.Dockerfile ``` -##### `/cnb/hooks.d/app.run.Dockerfile` +##### `/cnb/hooks.d/app.run.Dockerfile.out` ``` #!/bin/sh cat run.Dockerfile @@ -70,7 +87,7 @@ This example hook would allow a builder to install RPMs for each language runtim Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. -##### `/cnb/hooks.d/app.Dockerfile` +##### `/cnb/hooks.d/app.Dockerfile.out` ``` #!/bin/sh [[ -f Gemfile.lock ]] && cat /cnb/hooks.d/app.Dockerfile.d/Dockerfile-ruby @@ -107,7 +124,7 @@ USER ${CNB_USER_ID}:${CNB_GROUP_ID} COPY genpkgs /cnb/image/genpkgs ``` -run.Dockerfile for use with the example `app.Dockerfile` hook that always installs the latest version of curl: +`run.Dockerfile` for use with the example `app.run.Dockerfile.out` hook that always installs the latest version of curl: ``` ARG base_image FROM ${base_image} @@ -117,9 +134,9 @@ RUN echo ${build_id} RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* ``` -(note: this Dockerfile disables rebasing as OS package installation is not rebasable) +(note: this Dockerfile disables rebasing, as OS package installation is not rebasable) -run.Dockerfile for use with the example `app.Dockerfile` hook that installs a special package to /opt: +`run.Dockerfile` for use with the example `app.run.Dockerfile.out` hook that installs a special package to /opt: ``` ARG base_image FROM ${base_image} From 3534dc299a80d13151f03e94144369dfab9de29c Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 20 Jul 2021 15:51:51 -0400 Subject: [PATCH 007/372] RFC: Support Dockerfiles: fix typo Signed-off-by: Stephen Levine Co-authored-by: Daniel Mikusa --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 0e10f0e5c..d727c87ea 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -42,7 +42,7 @@ path = "./myhook" Each hook file path must be in the format `/cnb/hooks.d/.(build.|run.|)(.out|)`, where: -1. `build`, `run`, or empty specify the build-time base image, runtime base image, or both bash images, respectively. +1. `build`, `run`, or empty specify the build-time base image, runtime base image, or both base images, respectively. 2. The only valid format is `Dockerfile`, although support for, e.g. LLB JSON, could be added in the future. 3. If the `.out` suffix is present and the file is executable, the hook is executed in the context of the app directory, and its output must match ``. 4. If the `.out` suffix is not present, the contents of the file must match ``. From 05dc2d3de3de4470274b4b329c17ca5a08e3c092 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 21 Jul 2021 22:28:10 -0400 Subject: [PATCH 008/372] RFC: Support Dockerfiles: Buildpack-associated hooks Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 84 +++++++++++++++++++++++++++++++++------- 1 file changed, 71 insertions(+), 13 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index d727c87ea..bd2e0b3ca 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -22,22 +22,28 @@ Relying on Dockerfiles for base image generation and manipulation allows us to a # What it is [what-it-is]: #what-it-is -This RFC proposes that we replace stackpacks with multi-purpose build-time and runtime Dockerfiles. +This RFC proposes that we replace stackpacks with build-time and runtime Dockerfiles that act as pre-build hooks. +These hooks would be equivalent to base image / builder customizations, but would execute immediately before a build. + +Most importantly: the changes below do not impact buildpacks or the buildpack API. +For a given application, a build that uses hooks could be optimized by creating a more narrowly-scoped builder with the hooks pre-applied, without modifying the buildpacks. # How it Works [how-it-works]: #how-it-works Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. -### Builder-specified Dockerfiles +### Dynamically-applied Dockerfiles -A builder image may include any number of "hook" files in `/cnb/hooks.d/`. +A builder image may include any number of "hook" files in `/cnb/hooks.d/`. Hooks may be associated with zero or more buildpack IDs. The `pack create-builder` command is used to copy hooks into the builder image via `builder.toml`. E.g., ```toml [[hooks]] name = "app-hook" path = "./myhook" +buildpacks = ["com.example.buildpack1"] ``` + (Note: this format should not be considered final. Additional fields will be required to mark the format, target, etc. of the hook. These may be defined in subsequent spec PRs / sub-team RFCs.) Each hook file path must be in the format `/cnb/hooks.d/.(build.|run.|)(.out|)`, where: @@ -47,14 +53,19 @@ Each hook file path must be in the format `/cnb/hooks.d/.(build.|run.|)`. 4. If the `.out` suffix is not present, the contents of the file must match ``. -Hook files are evaluated (either read or executed) and the resulting Dockerfiles are applied to the build-time and runtime base images in lexographical order by ``. -Executable hook files must not modify the app directory when executed and may be executed in parallel. -However, the resulting Dockerfiles must not be applied in parallel. -A Dockerfile intended for the runtime base image is applied after the detection phase. -A Dockerfile intended for the build-time base image is applied before the detection phase. +Additionally, each hook file may be accompanied by an additional file of the same name, but with the suffix `.buildpacks`. +This file contains a newline-separated list of buildpack IDs. +If present, the hook is ignored unless one of the buildpacks in the list is selected during the buildpack detection phase. + +Hook files are evaluated (either read or executed) after the detection phase. +All Dockerfiles are applied to base images after the hooks are evaluated and before the buildpack build phase. +The resulting Dockerfiles are applied to the build-time and runtime base images in lexographical order by ``. + +Executable hook files must not modify the app directory when executed and may be evaluated in parallel. +However, the resulting Dockerfiles must not be applied in parallel to the same base image. If an executable hook exits with a non-zero status value, the build fails. -If a executable hook exits with a zero status value and no output, the hook is ignored. +If an executable hook exits with a zero status value and no output, the hook is ignored. Directories are ignored. Files at the top-level of `/cnb/hooks.d/` that do not match the specified file name format result in a build failure. @@ -81,9 +92,9 @@ cat build.Dockerfile cat run.Dockerfile ``` -#### Example: RPM Dockerfile Hook +#### Example: RPM Dockerfile Hook (app-based) -This example hook would allow a builder to install RPMs for each language runtime. +This example hook would allow a builder to install RPMs for each language runtime, based on the app directory. Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. @@ -94,9 +105,43 @@ Note: The Dockerfiles referenced must disable rebasing, and build times will be [[ -f package.json ]] && cat /cnb/hooks.d/app.Dockerfile.d/Dockerfile-node ``` -### Platform-specified Dockerfiles +#### Example: RPM Dockerfile Hook (buildpack-based) + +This example hook would allow a builder to install RPMs for each language runtime, based on the available buildpacks. + +Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. + -The same Dockerfile format may be used to create new base images or modify existing base images outside of the app build process. Any specified labels override existing values. +##### `/cnb/hooks.d/ruby.Dockerfile` +``` +... +LABEL io.buildpacks.rebasable=false # default, not needed +RUN yum install ruby +... +``` + +##### `/cnb/hooks.d/ruby.Dockerfile.buildpacks` +``` +com.example.ruby +``` + +##### `/cnb/hooks.d/node.Dockerfile` +``` +... +LABEL io.buildpacks.rebasable=false # default, not needed +RUN yum install nodejs +... +``` + +##### `/cnb/hooks.d/node.Dockerfile.buildpacks` +``` +com.example.node +``` + + +### Dockerfiles for Base Images + +The same Dockerfile format may be used to create new base images or modify existing base images outside of the app build process (e.g., before creating a builder). Any specified labels override existing values. Dockerfiles that are used to create a base image must create a `/cnb/image/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after all Dockerfiles are applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. @@ -148,6 +193,17 @@ RUN curl -L https://example.com/mypkg-install | sh # installs to /opt ``` (note: rebasing is explicitly allowed because only a single directory in /opt is created) +### Future: Dynamic Run-time Base Image Selection + +The outcomes targeted by https://github.com/buildpacks/rfcs/pull/174 could be achieved with an additional type of hook. + +For example, we could introduce a `ref` format that replaces the image instead of extending it: +``` +/cnb/hooks.d/myhook.run.ref # contains: index.docker.io/example/my-ruby-run-image +/cnb/hooks.d/myhook.run.buildpacks # contains: com.example.ruby-buildpack +``` + +This would simplify buildpacks by removing the need for PURLs outside of SBoM generation. The `packages` table could be removed from both buildpack.toml and `/bin/detect`. # Drawbacks [drawbacks]: #drawbacks @@ -158,6 +214,8 @@ RUN curl -L https://example.com/mypkg-install | sh # installs to /opt [alternatives]: #alternatives - Stackpacks +- "Stackpacks-lite" -- Instead of `.buildpacks`, have hooks participate in buildpack detection (provide-only, must come first) +- Use directories for hooks: `/cnb/hooks.d/app.build.Dockerfile.out` -> `/cnb/hooks.d/app/build.Dockerfile.out` # Unresolved Questions [unresolved-questions]: #unresolved-questions From f3e84c6220421900fc4499333f0fd09cd36f0d2e Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 22 Jul 2021 21:23:35 -0400 Subject: [PATCH 009/372] RFC: Support Dockerfiles: clarify builder vs. build-image Signed-off-by: Stephen Levine Co-authored-by: Emily Casey --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index bd2e0b3ca..06819a814 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -48,7 +48,7 @@ buildpacks = ["com.example.buildpack1"] Each hook file path must be in the format `/cnb/hooks.d/.(build.|run.|)(.out|)`, where: -1. `build`, `run`, or empty specify the build-time base image, runtime base image, or both base images, respectively. +1. `build`, `run`, or empty specify the builder image, runtime base image, or both base images, respectively. 2. The only valid format is `Dockerfile`, although support for, e.g. LLB JSON, could be added in the future. 3. If the `.out` suffix is present and the file is executable, the hook is executed in the context of the app directory, and its output must match ``. 4. If the `.out` suffix is not present, the contents of the file must match ``. From 1e813b987ad6734169ed7c74ac60f51f943dd74f Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 29 Jul 2021 01:44:31 -0400 Subject: [PATCH 010/372] RFC: Support Dockerfiles: integrate with buildpack API Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 159 +++++++++++++++++---------------------- 1 file changed, 70 insertions(+), 89 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 06819a814..43e8fca07 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -12,21 +12,20 @@ # Summary [summary]: #summary -This RFC introduces functionality for customizing base images, as an alternative to stackpacks. +This RFC introduces functionality for customizing base images, as an alternative to stackpacks ([RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md)). # Motivation [motivation]: #motivation -Relying on Dockerfiles for base image generation and manipulation allows us to apply buildpacks only to the problem that they solve best: managing application runtimes and dependencies. +[RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md) introduces complexity by defining an API that allows buildpacks to modify base images. To avoid this complexity, we could rely on generated Dockerfiles for base image manipulation. This would simplify the original proposal by, e.g., only requiring a copy of the buildpack on the build-time base image. # What it is [what-it-is]: #what-it-is -This RFC proposes that we replace stackpacks with build-time and runtime Dockerfiles that act as pre-build hooks. -These hooks would be equivalent to base image / builder customizations, but would execute immediately before a build. +This RFC proposes that we replace stackpacks with dynamically-generated build-time and runtime Dockerfiles that act as pre-build hooks. +These hooks would participate in detection and execute before the buildpack build process. -Most importantly: the changes below do not impact buildpacks or the buildpack API. -For a given application, a build that uses hooks could be optimized by creating a more narrowly-scoped builder with the hooks pre-applied, without modifying the buildpacks. +For a given application, a build that uses hooks could be optimized by creating a more narrowly-scoped builder that does not contain hooks. # How it Works [how-it-works]: #how-it-works @@ -35,61 +34,88 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap ### Dynamically-applied Dockerfiles -A builder image may include any number of "hook" files in `/cnb/hooks.d/`. Hooks may be associated with zero or more buildpack IDs. -The `pack create-builder` command is used to copy hooks into the builder image via `builder.toml`. E.g., +A builder image may include any number of "hook" directories in `/cnb/hooks/`. + +Hooks are similar to buildpacks: they have a `/bin/build` and `/bin/detect` executable. +However, instead of a `buildpack.toml` file, hooks have a `hook.toml` file: ```toml -[[hooks]] -name = "app-hook" -path = "./myhook" -buildpacks = ["com.example.buildpack1"] +api = "" + +[hook] +id = "" +name = "" +version = "" +homepage = "" +description = "" +keywords = [ "" ] + +[[hook.licenses]] +type = "" +uri = "" ``` -(Note: this format should not be considered final. Additional fields will be required to mark the format, target, etc. of the hook. These may be defined in subsequent spec PRs / sub-team RFCs.) +Hooks may be packaged and examined similar to buildpacks, but with analogous `pack hook` subcommands. + +Other pack CLI commands, such as `pack builder create`, will be extended to include support for hooks. -Each hook file path must be in the format `/cnb/hooks.d/.(build.|run.|)(.out|)`, where: +Unlike buildpacks, +- Hooks must not be included in a meta-buildpacks +- Hooks must not have `order`/`group` definitions in `hook.toml` -1. `build`, `run`, or empty specify the builder image, runtime base image, or both base images, respectively. -2. The only valid format is `Dockerfile`, although support for, e.g. LLB JSON, could be added in the future. -3. If the `.out` suffix is present and the file is executable, the hook is executed in the context of the app directory, and its output must match ``. -4. If the `.out` suffix is not present, the contents of the file must match ``. +Hooks participate in the buildpack detection process, with the same interface for `/bin/detect`. +However, +- `/bin/detect` is optional for hooks, and they are assumed to pass detection when it is not presetn. +- Hooks may only output `provides` entries to the build plan. They must not output `requires`. +- Hooks must all proceed regular buildpacks in `order` definitions (e.g., in `builder.toml`). +- Hooks are always `optional`. -Additionally, each hook file may be accompanied by an additional file of the same name, but with the suffix `.buildpacks`. -This file contains a newline-separated list of buildpack IDs. -If present, the hook is ignored unless one of the buildpacks in the list is selected during the buildpack detection phase. +Hooks generate Dockerfiles before the regular buildpack build phase. +To generate these Dockerfiles, the lifecycle executes the hook's `/bin/build` executable with the same interface as regular buildpacks. +However, +- Hooks `/bin/build` must not write to the app directory. +- Hooks `/bin/build` may be executed in parallel. +- Hooks `` directory is replaced by an `` directory. +- If a hook is missing `/bin/build`, the hook root is treated as a pre-populated `` directory. -Hook files are evaluated (either read or executed) after the detection phase. -All Dockerfiles are applied to base images after the hooks are evaluated and before the buildpack build phase. -The resulting Dockerfiles are applied to the build-time and runtime base images in lexographical order by ``. +After `/bin/build` executes, the `` directory may contain +- `build.toml`, with the same contents as a buildpack's `build.toml` +- Either `Dockerfile` or either or both of `build.Dockerfile` and `run.Dockerfile` -Executable hook files must not modify the app directory when executed and may be evaluated in parallel. -However, the resulting Dockerfiles must not be applied in parallel to the same base image. +Support for other instruction formats, e.g., LLB JSON files, could be added in the future. -If an executable hook exits with a non-zero status value, the build fails. -If an executable hook exits with a zero status value and no output, the hook is ignored. -Directories are ignored. -Files at the top-level of `/cnb/hooks.d/` that do not match the specified file name format result in a build failure. +`build.Dockerfile`, `run.Dockerfile`, and `Dockerfile` target the builder image, runtime base image, or both base images, respectively. -Both Dockerfiles must accept `base_image` and `build_id` args. -The `base_image` arg allows the lifecycle to specify the original base image. -The `build_id` arg allows the app developer to bust the cache after a certain layer and must be defaulted to `0`. When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). +If no Dockerfiles are present, `/bin/build` may still consume build plan entries and add metadata to `build.toml`. + +Dockerfiles are applied to their corresponding base images after all hooks are executed and before any regular buildpacks are executed. +Dockerfiles are applied in the order determined during buildpack detection. + +All Dockerfiles are provided with `base_image` and `build_id` args. +The `base_image` arg allows the Dockerfile to reference the original base image. +The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. +When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. +Finally, base images may be statically labeled with any number of `provides` that are treated as build plan entries. +These `provides` may contain fields other than `name`, which, when mismatched with `requires`, mark the entry as `unmet` by the stack. +This is important to ensure that: +- Rebasing always remains an option for end users. +- Buildpacks do not become dependent on hooks. +- Builds can be time-optimized by creating base images ahead of time. + +NOTE: the above can be accomplished by a hook with a no-op `/bin/build` -- do we really need this? #### Example: App-specified Dockerfile Hook This example hook would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." The app developer can decide whether the extensions are rebasable. -##### `/cnb/hooks.d/app.build.Dockerfile.out` -``` -#!/bin/sh -cat build.Dockerfile -``` -##### `/cnb/hooks.d/app.run.Dockerfile.out` +##### `/cnb/hooks/com.example.apphook/bin/build` ``` #!/bin/sh -cat run.Dockerfile +[ -f build.Dockerfile ] && cp build.Dockerfile "$1/" +[ -f run.Dockerfile ] && cp run.Dockerfile "$1/" ``` #### Example: RPM Dockerfile Hook (app-based) @@ -98,44 +124,11 @@ This example hook would allow a builder to install RPMs for each language runtim Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. -##### `/cnb/hooks.d/app.Dockerfile.out` +##### `/cnb/hooks/com.example.rpmhook/bin/build` ``` #!/bin/sh -[[ -f Gemfile.lock ]] && cat /cnb/hooks.d/app.Dockerfile.d/Dockerfile-ruby -[[ -f package.json ]] && cat /cnb/hooks.d/app.Dockerfile.d/Dockerfile-node -``` - -#### Example: RPM Dockerfile Hook (buildpack-based) - -This example hook would allow a builder to install RPMs for each language runtime, based on the available buildpacks. - -Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. - - -##### `/cnb/hooks.d/ruby.Dockerfile` -``` -... -LABEL io.buildpacks.rebasable=false # default, not needed -RUN yum install ruby -... -``` - -##### `/cnb/hooks.d/ruby.Dockerfile.buildpacks` -``` -com.example.ruby -``` - -##### `/cnb/hooks.d/node.Dockerfile` -``` -... -LABEL io.buildpacks.rebasable=false # default, not needed -RUN yum install nodejs -... -``` - -##### `/cnb/hooks.d/node.Dockerfile.buildpacks` -``` -com.example.node +[ -f Gemfile.lock ] && cp "$BUILDPACK_DIR/Dockerfile-ruby" "$1/Dockerfile" +[ -f package.json ] && cp "$BUILDPACK_DIR/Dockerfile-node" "$1/Dockerfile" ``` @@ -193,17 +186,6 @@ RUN curl -L https://example.com/mypkg-install | sh # installs to /opt ``` (note: rebasing is explicitly allowed because only a single directory in /opt is created) -### Future: Dynamic Run-time Base Image Selection - -The outcomes targeted by https://github.com/buildpacks/rfcs/pull/174 could be achieved with an additional type of hook. - -For example, we could introduce a `ref` format that replaces the image instead of extending it: -``` -/cnb/hooks.d/myhook.run.ref # contains: index.docker.io/example/my-ruby-run-image -/cnb/hooks.d/myhook.run.buildpacks # contains: com.example.ruby-buildpack -``` - -This would simplify buildpacks by removing the need for PURLs outside of SBoM generation. The `packages` table could be removed from both buildpack.toml and `/bin/detect`. # Drawbacks [drawbacks]: #drawbacks @@ -214,8 +196,7 @@ This would simplify buildpacks by removing the need for PURLs outside of SBoM ge [alternatives]: #alternatives - Stackpacks -- "Stackpacks-lite" -- Instead of `.buildpacks`, have hooks participate in buildpack detection (provide-only, must come first) -- Use directories for hooks: `/cnb/hooks.d/app.build.Dockerfile.out` -> `/cnb/hooks.d/app/build.Dockerfile.out` +- Previous versions of this RFC that don't interact with the buildpack API. # Unresolved Questions [unresolved-questions]: #unresolved-questions From ba9ebd7d619a60264bb117e4d69c9dbae2cfaf49 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 29 Jul 2021 18:52:35 -0400 Subject: [PATCH 011/372] RFC: Support Dockerfiles: Add build args + runtime SBoM Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 43e8fca07..9086a003b 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -78,14 +78,20 @@ However, - If a hook is missing `/bin/build`, the hook root is treated as a pre-populated `` directory. After `/bin/build` executes, the `` directory may contain -- `build.toml`, with the same contents as a buildpack's `build.toml` +- `build.toml`, with the same contents as a normal buildpack's `build.toml`, but + - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` or `Dockerfile` +- `launch.toml`, with the same contents as a normal buildpack's `launch.toml`, but + - Without the `processes` table array + - Without the `slices` table array + - With an additional `args` table array with `name` and `value` fields that are provided as build args to `run.Dockerfile` or `Dockerfile` + - Either `Dockerfile` or either or both of `build.Dockerfile` and `run.Dockerfile` Support for other instruction formats, e.g., LLB JSON files, could be added in the future. `build.Dockerfile`, `run.Dockerfile`, and `Dockerfile` target the builder image, runtime base image, or both base images, respectively. -If no Dockerfiles are present, `/bin/build` may still consume build plan entries and add metadata to `build.toml`. +If no Dockerfiles are present, `/bin/build` may still consume build plan entries and add metadata to `build.toml`/`launch.toml`. Dockerfiles are applied to their corresponding base images after all hooks are executed and before any regular buildpacks are executed. Dockerfiles are applied in the order determined during buildpack detection. @@ -95,6 +101,9 @@ The `base_image` arg allows the Dockerfile to reference the original base image. The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). +Build args specified in `build.toml` are provided to `build.Dockerfile` or `Dockerfile` (when applied to the build-time base image). +Build args specified in `launch.toml` are provided to `run.Dockerfile` or `Dockerfile` (when applied to the runtime base image). + A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. Finally, base images may be statically labeled with any number of `provides` that are treated as build plan entries. From 58ed531a97e1196259ba74e95a9370dcf61f70ee Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 9 Sep 2021 14:34:49 -0400 Subject: [PATCH 012/372] Fix buildpack dir env var Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 9086a003b..bb3ea7b56 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -136,8 +136,8 @@ Note: The Dockerfiles referenced must disable rebasing, and build times will be ##### `/cnb/hooks/com.example.rpmhook/bin/build` ``` #!/bin/sh -[ -f Gemfile.lock ] && cp "$BUILDPACK_DIR/Dockerfile-ruby" "$1/Dockerfile" -[ -f package.json ] && cp "$BUILDPACK_DIR/Dockerfile-node" "$1/Dockerfile" +[ -f Gemfile.lock ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-ruby" "$1/Dockerfile" +[ -f package.json ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-node" "$1/Dockerfile" ``` From ebf6b2a9fc1f093604741407e291deef1506d1cd Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 9 Sep 2021 14:35:33 -0400 Subject: [PATCH 013/372] Clarify detect logic Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index bb3ea7b56..d70f293df 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -64,7 +64,7 @@ Unlike buildpacks, Hooks participate in the buildpack detection process, with the same interface for `/bin/detect`. However, -- `/bin/detect` is optional for hooks, and they are assumed to pass detection when it is not presetn. +- `/bin/detect` is optional for hooks, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. - Hooks may only output `provides` entries to the build plan. They must not output `requires`. - Hooks must all proceed regular buildpacks in `order` definitions (e.g., in `builder.toml`). - Hooks are always `optional`. From 22f0ef4ad7135fd49831eef2d2ab1221c5b56dad Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Thu, 9 Sep 2021 14:41:28 -0400 Subject: [PATCH 014/372] Fix typo Signed-off-by: Stephen Levine Co-authored-by: Javier Romero --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index d70f293df..643675ea2 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -99,7 +99,7 @@ Dockerfiles are applied in the order determined during buildpack detection. All Dockerfiles are provided with `base_image` and `build_id` args. The `base_image` arg allows the Dockerfile to reference the original base image. The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. -When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layerrs will be rebuilt on the next build (as the value will change). +When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layers will be rebuilt on the next build (as the value will change). Build args specified in `build.toml` are provided to `build.Dockerfile` or `Dockerfile` (when applied to the build-time base image). Build args specified in `launch.toml` are provided to `run.Dockerfile` or `Dockerfile` (when applied to the runtime base image). From 084822e9611d46349d008c9cd64476016f614b25 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Fri, 10 Sep 2021 13:18:03 -0400 Subject: [PATCH 015/372] Use UUID for build_id Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 643675ea2..bbbf5a19d 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -98,7 +98,7 @@ Dockerfiles are applied in the order determined during buildpack detection. All Dockerfiles are provided with `base_image` and `build_id` args. The `base_image` arg allows the Dockerfile to reference the original base image. -The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. +The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. The executor of the Dockerfile will provide the `build_id` as a UUID (this eliminates the need to track this variable). When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layers will be rebuilt on the next build (as the value will change). Build args specified in `build.toml` are provided to `build.Dockerfile` or `Dockerfile` (when applied to the build-time base image). From b3b38ea37a3e13fffb7305d9a5b6ad9c6013a18f Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Fri, 10 Sep 2021 13:19:08 -0400 Subject: [PATCH 016/372] Fix wording re: rebasable label Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index bbbf5a19d..442973880 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -104,7 +104,7 @@ When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent la Build args specified in `build.toml` are provided to `build.Dockerfile` or `Dockerfile` (when applied to the build-time base image). Build args specified in `launch.toml` are provided to `run.Dockerfile` or `Dockerfile` (when applied to the runtime base image). -A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. +A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically on the base image before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. If multiple Dockerfiles are applied, all must set `io.buildpacks.rebasable=true` for the final value to be `true`. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. Finally, base images may be statically labeled with any number of `provides` that are treated as build plan entries. These `provides` may contain fields other than `name`, which, when mismatched with `requires`, mark the entry as `unmet` by the stack. From 6eed4b299c6a6858e81a7e2d04e7a34125a8e059 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 28 Sep 2021 22:58:41 -0400 Subject: [PATCH 017/372] Clarify phases Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 442973880..654628b5e 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -30,7 +30,15 @@ For a given application, a build that uses hooks could be optimized by creating # How it Works [how-it-works]: #how-it-works -Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. +Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. The order of operations would be something like the following: +* analyze +* detect +* restore +* run hooks' bin/build, output Dockerfiles are written to a volume +* apply Dockerfiles to run image (could run in parallel with below two) +* apply Dockerfiles to build image +* build +* export ### Dynamically-applied Dockerfiles From fde7b84d0db8864292a5868a8bce39317df2f9a3 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 28 Sep 2021 23:05:11 -0400 Subject: [PATCH 018/372] RFC: Support Dockerfiles: clarify UID/GID of executables Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 654628b5e..8c71b1fe6 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -70,7 +70,7 @@ Unlike buildpacks, - Hooks must not be included in a meta-buildpacks - Hooks must not have `order`/`group` definitions in `hook.toml` -Hooks participate in the buildpack detection process, with the same interface for `/bin/detect`. +Hooks participate in the buildpack detection process, with the same UID, GID, and interface for `/bin/detect`. However, - `/bin/detect` is optional for hooks, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. - Hooks may only output `provides` entries to the build plan. They must not output `requires`. @@ -78,7 +78,7 @@ However, - Hooks are always `optional`. Hooks generate Dockerfiles before the regular buildpack build phase. -To generate these Dockerfiles, the lifecycle executes the hook's `/bin/build` executable with the same interface as regular buildpacks. +To generate these Dockerfiles, the lifecycle executes the hook's `/bin/build` executable with the same UID, GID, and interface as regular buildpacks. However, - Hooks `/bin/build` must not write to the app directory. - Hooks `/bin/build` may be executed in parallel. From f643f8b9855f6d78dac96d1e663ef19b03ba1af7 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 28 Sep 2021 23:10:56 -0400 Subject: [PATCH 019/372] RFC: Support Dockerfiles: rename hook to extension Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 80 ++++++++++++++++++++-------------------- 1 file changed, 40 insertions(+), 40 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 8c71b1fe6..e270ad1e2 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -22,10 +22,10 @@ This RFC introduces functionality for customizing base images, as an alternative # What it is [what-it-is]: #what-it-is -This RFC proposes that we replace stackpacks with dynamically-generated build-time and runtime Dockerfiles that act as pre-build hooks. -These hooks would participate in detection and execute before the buildpack build process. +This RFC proposes that we replace stackpacks with dynamically-generated build-time and runtime Dockerfiles that act as pre-build base image extensions. +These extensions would participate in detection and execute before the buildpack build process. -For a given application, a build that uses hooks could be optimized by creating a more narrowly-scoped builder that does not contain hooks. +For a given application, a build that uses extensions could be optimized by creating a more narrowly-scoped builder that does not contain extensions. # How it Works [how-it-works]: #how-it-works @@ -34,7 +34,7 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap * analyze * detect * restore -* run hooks' bin/build, output Dockerfiles are written to a volume +* run extensions' bin/build, output Dockerfiles are written to a volume * apply Dockerfiles to run image (could run in parallel with below two) * apply Dockerfiles to build image * build @@ -42,48 +42,48 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap ### Dynamically-applied Dockerfiles -A builder image may include any number of "hook" directories in `/cnb/hooks/`. +A builder image may include any number of "extensions" directories in `/cnb/ext/`. -Hooks are similar to buildpacks: they have a `/bin/build` and `/bin/detect` executable. -However, instead of a `buildpack.toml` file, hooks have a `hook.toml` file: +Extensions are similar to buildpacks: they have a `/bin/build` and `/bin/detect` executable. +However, instead of a `buildpack.toml` file, extensions have a `extension.toml` file: ```toml api = "" -[hook] -id = "" -name = "" -version = "" -homepage = "" -description = "" +[extension] +id = "" +name = "" +version = "" +homepage = "" +description = "" keywords = [ "" ] -[[hook.licenses]] +[[extension.licenses]] type = "" uri = "" ``` -Hooks may be packaged and examined similar to buildpacks, but with analogous `pack hook` subcommands. +Extensions may be packaged and examined similar to buildpacks, but with analogous `pack extension` subcommands. -Other pack CLI commands, such as `pack builder create`, will be extended to include support for hooks. +Other pack CLI commands, such as `pack builder create`, will be extended to include support for extensions. Unlike buildpacks, -- Hooks must not be included in a meta-buildpacks -- Hooks must not have `order`/`group` definitions in `hook.toml` +- Extensions must not be included in a meta-buildpacks +- Extensions must not have `order`/`group` definitions in `extension.toml` -Hooks participate in the buildpack detection process, with the same UID, GID, and interface for `/bin/detect`. +Extensions participate in the buildpack detection process, with the same UID, GID, and interface for `/bin/detect`. However, -- `/bin/detect` is optional for hooks, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. -- Hooks may only output `provides` entries to the build plan. They must not output `requires`. -- Hooks must all proceed regular buildpacks in `order` definitions (e.g., in `builder.toml`). -- Hooks are always `optional`. +- `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. +- Extensions may only output `provides` entries to the build plan. They must not output `requires`. +- Extensions must all proceed regular buildpacks in `order` definitions (e.g., in `builder.toml`). +- Extensions are always `optional`. -Hooks generate Dockerfiles before the regular buildpack build phase. -To generate these Dockerfiles, the lifecycle executes the hook's `/bin/build` executable with the same UID, GID, and interface as regular buildpacks. +Extensions generate Dockerfiles before the regular buildpack build phase. +To generate these Dockerfiles, the lifecycle executes the extension's `/bin/build` executable with the same UID, GID, and interface as regular buildpacks. However, -- Hooks `/bin/build` must not write to the app directory. -- Hooks `/bin/build` may be executed in parallel. -- Hooks `` directory is replaced by an `` directory. -- If a hook is missing `/bin/build`, the hook root is treated as a pre-populated `` directory. +- Extensions `/bin/build` must not write to the app directory. +- Extensions `/bin/build` may be executed in parallel. +- Extensions `` directory is replaced by an `` directory. +- If an extension is missing `/bin/build`, the extension root is treated as a pre-populated `` directory. After `/bin/build` executes, the `` directory may contain - `build.toml`, with the same contents as a normal buildpack's `build.toml`, but @@ -101,7 +101,7 @@ Support for other instruction formats, e.g., LLB JSON files, could be added in t If no Dockerfiles are present, `/bin/build` may still consume build plan entries and add metadata to `build.toml`/`launch.toml`. -Dockerfiles are applied to their corresponding base images after all hooks are executed and before any regular buildpacks are executed. +Dockerfiles are applied to their corresponding base images after all extensions are executed and before any regular buildpacks are executed. Dockerfiles are applied in the order determined during buildpack detection. All Dockerfiles are provided with `base_image` and `build_id` args. @@ -118,30 +118,30 @@ Finally, base images may be statically labeled with any number of `provides` tha These `provides` may contain fields other than `name`, which, when mismatched with `requires`, mark the entry as `unmet` by the stack. This is important to ensure that: - Rebasing always remains an option for end users. -- Buildpacks do not become dependent on hooks. +- Buildpacks do not become dependent on extensions. - Builds can be time-optimized by creating base images ahead of time. -NOTE: the above can be accomplished by a hook with a no-op `/bin/build` -- do we really need this? +NOTE: the above can be accomplished by an extension with a no-op `/bin/build` -- do we really need this? -#### Example: App-specified Dockerfile Hook +#### Example: App-specified Dockerfile Extension -This example hook would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." +This example extension would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." The app developer can decide whether the extensions are rebasable. -##### `/cnb/hooks/com.example.apphook/bin/build` +##### `/cnb/ext/com.example.appext/bin/build` ``` #!/bin/sh [ -f build.Dockerfile ] && cp build.Dockerfile "$1/" [ -f run.Dockerfile ] && cp run.Dockerfile "$1/" ``` -#### Example: RPM Dockerfile Hook (app-based) +#### Example: RPM Dockerfile Extension (app-based) -This example hook would allow a builder to install RPMs for each language runtime, based on the app directory. +This example extension would allow a builder to install RPMs for each language runtime, based on the app directory. Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. -##### `/cnb/hooks/com.example.rpmhook/bin/build` +##### `/cnb/ext/com.example.rpmext/bin/build` ``` #!/bin/sh [ -f Gemfile.lock ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-ruby" "$1/Dockerfile" @@ -179,7 +179,7 @@ USER ${CNB_USER_ID}:${CNB_GROUP_ID} COPY genpkgs /cnb/image/genpkgs ``` -`run.Dockerfile` for use with the example `app.run.Dockerfile.out` hook that always installs the latest version of curl: +`run.Dockerfile` for use with the example `app.run.Dockerfile.out` extension that always installs the latest version of curl: ``` ARG base_image FROM ${base_image} @@ -191,7 +191,7 @@ RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* ``` (note: this Dockerfile disables rebasing, as OS package installation is not rebasable) -`run.Dockerfile` for use with the example `app.run.Dockerfile.out` hook that installs a special package to /opt: +`run.Dockerfile` for use with the example `app.run.Dockerfile.out` extension that installs a special package to /opt: ``` ARG base_image FROM ${base_image} From 3a37d5483c8066a55f5cbf6c7a2c2968aa3b48bb Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 28 Sep 2021 23:17:24 -0400 Subject: [PATCH 020/372] RFC: Support Dockerfiles: fix typo Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index e270ad1e2..17a48676f 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -74,7 +74,7 @@ Extensions participate in the buildpack detection process, with the same UID, GI However, - `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. - Extensions may only output `provides` entries to the build plan. They must not output `requires`. -- Extensions must all proceed regular buildpacks in `order` definitions (e.g., in `builder.toml`). +- Extensions must all precede regular buildpacks in `order` definitions (e.g., in `builder.toml`). - Extensions are always `optional`. Extensions generate Dockerfiles before the regular buildpack build phase. From f513c24dd381e285b92c4aa6504bd4bcca74114c Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 28 Sep 2021 23:27:18 -0400 Subject: [PATCH 021/372] RFC: Support Dockerfiles: remove static provides Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 17a48676f..22b1eeabc 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -114,15 +114,6 @@ Build args specified in `launch.toml` are provided to `run.Dockerfile` or `Docke A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically on the base image before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. If multiple Dockerfiles are applied, all must set `io.buildpacks.rebasable=true` for the final value to be `true`. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. -Finally, base images may be statically labeled with any number of `provides` that are treated as build plan entries. -These `provides` may contain fields other than `name`, which, when mismatched with `requires`, mark the entry as `unmet` by the stack. -This is important to ensure that: -- Rebasing always remains an option for end users. -- Buildpacks do not become dependent on extensions. -- Builds can be time-optimized by creating base images ahead of time. - -NOTE: the above can be accomplished by an extension with a no-op `/bin/build` -- do we really need this? - #### Example: App-specified Dockerfile Extension This example extension would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." @@ -219,6 +210,7 @@ RUN curl -L https://example.com/mypkg-install | sh # installs to /opt [unresolved-questions]: #unresolved-questions - Should `genpkgs` be part of this proposal? Opinion: Yes, otherwise it's difficult to maintain a valid SBoM. +- Should we allow base images to provide build plan entries so that extensions aren't required to satisfy buildpacks? Opinion: Not yet, no-op extension can be used for now. # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From cd175bbea36217de92effbd92f79f064bf7e48fd Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 18 Nov 2021 15:49:25 -0600 Subject: [PATCH 022/372] RFC: Replace positional args to Buildpack executables with env vars Signed-off-by: Joe Kutner --- text/0000-buildpack-input-vars.md | 142 ++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 text/0000-buildpack-input-vars.md diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md new file mode 100644 index 000000000..ef3e7d1e4 --- /dev/null +++ b/text/0000-buildpack-input-vars.md @@ -0,0 +1,142 @@ +# Meta +[meta]: #meta +- Name: Replace positional args to Buildpack executables with env vars +- Start Date: 2021-11-18 +- Author(s): [jkutner](https://github.com/jkutner) +- Status: Draft Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This is a proposal to replace the positional arguments of Buildpack executables with named environment variables. + +# Definitions +[definitions]: #definitions + +- *positional arguments* - values passed into an executable command such that they are accessible as `$1`, `$2`, etc + +# Motivation +[motivation]: #motivation + +Postional arguments to Buildpack executables have been the way for the buildpack execution engine to provide inputs for buildpacks since buildpacks v1 was created. However, positional arguments are limiting, and what they represent is not obvious without reading the spec. + +In the spec today they are defined as: + +* `/bin/detect ` +* `/bin/build ` + +Using env vars helps make these inputs more easily accesible from language bindings like [libcnb.bash](https://github.com/jkutner/libcnb.bash). + +# What it is +[what-it-is]: #what-it-is + +We will deprecate the positional arguments to `bin/detect` and `bin/build` with the following environment variables: + +### bin/detect + +* `CNB_PLATFORM_DIR` - replaces the first positional argument to `bin/build`. Uses the same env var name as the Platform spec. +* `CNB_PLAN_PATH` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. + +### bin/build + +* `CNB_BP_LAYERS_DIR` - replaces the first positional argument to `bin/build`. Uses `_BP_` to differentiate it from the `CNB_LAYERS_DIR` in the Plaform spec, which is a different value. +* `CNB_PLATFORM_DIR` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. +* `CNB_PLAN_PATH` - replaces the third positional argument to `bin/build`. Uses the same env var name as the Platform spec. + +# How it Works +[how-it-works]: #how-it-works + +Provide the environment variables with the same mechanism used to provide `CNB_BUILDPACK_DIR`. For example in [lifecycle's `build.go`](https://github.com/buildpacks/lifecycle/blob/880a801db2d4bfbb39671a66f7aadd96c0231e37/buildpack/build.go): + +```go +cmd.Env = append(cmd.Env, EnvPlatformDir+"="+b.Dir) +``` + +The positional arguments will be deprecated, but no warnings will be emitted if they are consumed. The lifecycle will continue to provide them to buildpack executable indefinitely, with no plan to remove them. + +# Drawbacks +[drawbacks]: #drawbacks + +- People have been using positional arguments to buildpacks for literally a decade + +# Alternatives +[alternatives]: #alternatives + +- Do this but don't deprecate the positional arguments (support both) + +# Prior Art +[prior-art]: #prior-art + +- Buildpack v1, v2a, & v2b + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +In the Buildpack spec: + +### Detection + +Executable: `/bin/detect`, Working Dir: `` + +| Input | Attributes | Description +|---------------------------|------------|---------------------------------------------- +| `$0` | | Absolute path of `/bin/detect` executable +| `$CNB_PLAN_PATH` | E | Absolute path to the build plan +| `$CNB_PLATFORM_DIR` | AR | Absolute path to the platform directory +| `$CNB_PLATFORM_DIR/env/` | | User-provided environment variables for build +| `$CNB_PLATFORM_DIR/#` | | Platform-specific extensions + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Pass (0), fail (100), or error (1-99, 101+) +| Standard output | Logs (info) +| Standard error | Logs (warnings, errors) +| `$CNB_PLAN_PATH` | Contributions to the the Build Plan (TOML) + +### Build + +Executable: `/bin/build`, Working Dir: `` + +| Input | Attributes | Description +|---------------------------|------------|---------------------------------- +| `$0` | | Absolute path of `/bin/build` executable +| `$CNB_BP_LAYERS_DIR` | EIC | Absolute path to the buildpack layers directory +| `$CNB_PLAN_PATH` | ER | Relevant [Buildpack Plan entries](#buildpack-plan-toml) from detection (TOML) +| `$CNB_PLATFORM_DIR` | AR | Absolute path to the platform directory +| `$CNB_PLATFORM_DIR/env/` | | User-provided environment variables for build +| `$CNB_PLATFORM_DIR/#` | | Platform-specific extensions + +| Output | Description +|------------------------------------------|-------------------------------------- +| [exit status] | Success (0) or failure (1+) +| Standard output | Logs (info) +| Standard error | Logs (warnings, errors) +| `$CNB_BP_LAYERS_DIR/launch.toml` | App metadata (see [launch.toml](#launchtoml-toml)) +| `$CNB_BP_LAYERS_DIR/launch.sbom.` | Launch Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) +| `$CNB_BP_LAYERS_DIR/build.toml` | Build metadata (see [build.toml](#buildtoml-toml)) +| `$CNB_BP_LAYERS_DIR/build.sbom.` | Build Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) +| `$CNB_BP_LAYERS_DIR/store.toml` | Persistent metadata (see [store.toml](#storetoml-toml)) +| `$CNB_BP_LAYERS_DIR/.toml` | Layer metadata (see [Layer Content Metadata](#layer-content-metadata-toml)) +| `$CNB_BP_LAYERS_DIR/.sbom.` | Layer Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) +| `$CNB_BP_LAYERS_DIR//bin/` | Binaries for launch and/or subsequent buildpacks +| `$CNB_BP_LAYERS_DIR//lib/` | Shared libraries for launch and/or subsequent buildpacks +| `$CNB_BP_LAYERS_DIR//profile.d/` | Scripts sourced by Bash before launch +| `$CNB_BP_LAYERS_DIR//profile.d//` | Scripts sourced by Bash before launch for a particular process type +| `$CNB_BP_LAYERS_DIR//exec.d/` | Executables that provide env vars via the [Exec.d Interface](#execd) before launch +| `$CNB_BP_LAYERS_DIR//exec.d//` | Executables that provide env vars for a particular process type via the [Exec.d Interface](#execd) before launch +| `$CNB_BP_LAYERS_DIR//include/` | C/C++ headers for subsequent buildpacks +| `$CNB_BP_LAYERS_DIR//pkgconfig/` | Search path for pkg-config for subsequent buildpacks +| `$CNB_BP_LAYERS_DIR//env/` | Env vars for launch and/or subsequent buildpacks +| `$CNB_BP_LAYERS_DIR//env.launch/` | Env vars for launch (after `env`, before `profile.d`) +| `$CNB_BP_LAYERS_DIR//env.launch//` | Env vars for launch (after `env`, before `profile.d`) for the launched process +| `$CNB_BP_LAYERS_DIR//env.build/` | Env vars for subsequent buildpacks (after `env`) +| `$CNB_BP_LAYERS_DIR//*` | Other content for launch and/or subsequent buildpacks From 60ce995d6978cd0d29883522bb920e3a32999f1e Mon Sep 17 00:00:00 2001 From: Daniel Mikusa Date: Mon, 13 Dec 2021 14:53:10 -0500 Subject: [PATCH 023/372] Add Migration section This PR proposes to add a Migration section to the RFC template. The purpose of this section is to ensure consideration is given to how an RFC might break existing APIs and workflows so that a discussion can be had in advance of an RFC being approved. As we get more and more users, this seems like an appropriate step to take to show that stability and continuity are important considerations as we evolve buildpacks. Signed-off-by: Daniel Mikusa --- 0000-template.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/0000-template.md b/0000-template.md index fa122548d..b4256f4e7 100644 --- a/0000-template.md +++ b/0000-template.md @@ -44,6 +44,11 @@ This is the technical portion of the RFC, where you explain the design in suffic The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. +# Migration +[migration]: #migration + +This section should document breaks to public API and breaks in compatibility due to this RFC's proposed changes. In addition, it should document the proposed steps that one would need to take to work through these changes. Care should be give to include all applicable personas, such as platform developers, buildpack developers, buildpack users and consumers of buildpack images. + # Drawbacks [drawbacks]: #drawbacks From 3c2aafdc6f4c3c226efb89e23f51d29534b6d4ce Mon Sep 17 00:00:00 2001 From: Anthony Emengo Date: Mon, 15 Nov 2021 17:56:14 -0500 Subject: [PATCH 024/372] Add state fields Signed-off-by: Anthony Emengo --- merge-rfc.sh | 2 +- text/0001-pack-suggest-stacks.md | 1 + text/0002-pack-logging-refactor.md | 1 + text/0003-pack-inspect-image.md | 1 + text/0004-rfc-process.md | 1 + text/0005-contractual-build-plan.md | 1 + text/0006-stage-specific-mixins.md | 1 + text/0007-spec-distribution.md | 1 + text/0008-buildpack-config-for-dist.md | 1 + text/0009-auto-load-user-provided-environment-variables.md | 1 + text/0010-api-versions.md | 1 + text/0011-lifecycle-descriptor.md | 1 + text/0012-service-binding.md | 1 + text/0013-app-layer-metadata-source.md | 1 + text/0014-combined-restorer-analyzer-phases.md | 1 + text/0015-windows-lifecycle.md | 1 + text/0016-use-email-more.md | 1 + text/0017-pack-build-default-process-flag.md | 1 + text/0018-remove-pack-run.md | 1 + text/0019-project-descriptor.md | 1 + text/0020-landing-page.md | 1 + text/0021-lifecycle-compat-verification.md | 1 + text/0022-client-side-buildpack-registry.md | 1 + text/0023-circleci-orb.md | 1 + text/0024-lifecycle-multicall-binary-for-build.md | 1 + text/0025-dont-trust-builders.md | 1 + text/0026-lifecycle-all.md | 1 + text/0027-spec-api-branches.md | 1 + text/0028-remove-cred-helpers.md | 1 + text/0029-template-changes.md | 1 + text/0030-links-for-buildpacks.md | 1 + text/0031-bionic-mixins.md | 1 + text/0032-update-json-cnb-registry.md | 1 + text/0033-add-author.md | 1 + text/0034-image-labels.md | 1 + text/0035-buildpack-versions.md | 1 + text/0036-cnb-buildpack-directory-env-var.md | 1 + text/0037-buildpack-uris.md | 1 + text/0038-gh-repo-labels.md | 1 + text/0039-release-process.md | 1 + text/0040-export-report.md | 1 + text/0041-api-version-compat.md | 1 + text/0042-process-specific-env.md | 1 + text/0043-increase-build-plan-flexibility.md | 1 + text/0044-pack-publish-buildpack.md | 1 + text/0045-launcher-arguments.md | 1 + text/0046-pack-pull-policy.md | 1 + text/0047-danger-zone.md | 1 + text/0048-inline-buildpack.md | 1 + text/0049-multi-api-lifecycle-descriptor.md | 1 + text/0050-stack-metadata.md | 1 + text/0051-override-env-by-default.md | 1 + text/0052-opt-in-layer-caching.md | 1 + text/0053-decouple-buildpack-plan-and-bom.md | 1 + text/0054-project-descriptor-schema.md | 1 + text/0055-deprecate-service-bindings.md | 1 + text/0056-any-stack-buildpacks.md | 1 + text/0057-exec.d-shell-free-profile-d.md | 1 + text/0058-pack-subcommands.md | 1 + text/0059-label-rfcs.md | 1 + text/0060-create-repo-issues.md | 1 + text/0061-relax-mixin-contract.md | 1 + text/0062-distribution-team.md | 1 + text/0063-stack-images-config-env-path-must-exist.md | 1 + text/0064-buildpacks-contribute-default-process-type.md | 1 + text/0065-builder-spec.md | 1 + ...66-lifecycle-prelease-version-and-experimental-section.md | 1 + text/0067-report-toml-image-manifest-size.md | 1 + text/0068-buildpack-registry-search-api.md | 1 + text/0069-stack-buildpacks.md | 1 + text/0070-new-buildpack-toml-keys.md | 1 + text/0071-cnb-user-research-2021.md | 1 + text/0072-image-workdir.md | 1 + text/0073-offline-buildpackages.md | 5 +++++ text/0074-layer-table.md | 1 + text/0075-move-analyze-phase.md | 1 + text/0076-windows-security-identifiers.md | 1 + text/0077-pack-buildpack-create.md | 1 + text/0078-group-additions.md | 1 + text/0079-create-stackify-repo.md | 1 + text/0080-builder-key-in-project-descriptor.md | 1 + text/0081-process-specific-working-directory.md | 1 + text/0082-pack-package-refactor.md | 1 + text/0083-best-practices-and-guidelines.md | 1 + text/0084-project-descriptor-domains.md | 1 + text/0084-rfc-issue-generation.md | 1 + text/0085-run-uid.md | 1 + text/0086-component-level-contrib.md | 1 + text/0087-bom-in-layer-metadata.md | 1 + text/0088-minimum-docs-lifecycle.md | 1 + text/0089-buildpack-authors-tooling-subteam.md | 1 + text/0090-intro-video-script.md | 1 + text/0091-pack-cache-options.md | 1 + text/0092-add-visual-pack-build.md | 1 + text/0093-remove-shell-processes.md | 1 + text/0094-add-status-field.md | 2 +- text/0095-sbom.md | 1 + text/0096-remove-stacks-mixins.md | 1 + 98 files changed, 102 insertions(+), 2 deletions(-) diff --git a/merge-rfc.sh b/merge-rfc.sh index 96b8d1930..54c9e1b0f 100755 --- a/merge-rfc.sh +++ b/merge-rfc.sh @@ -146,7 +146,7 @@ if [[ "$OSTYPE" == "darwin"* ]]; then fi sed $SEDOPTION "s|- RFC Pull Request:.*|- RFC Pull Request: [${REPO}#${PR_NUMBER}](https://github.com/${OWNER}/${REPO}/pull/${PR_NUMBER})|" "${SOURCE_DOC}" sed $SEDOPTION "s|- CNB Issue:.*|- CNB Issue: $ISSUES_TEXT|" "${SOURCE_DOC}" -sed $SEDOPTION "s|- State:.*|- State: **Approved**|" "${SOURCE_DOC}" +sed $SEDOPTION "s|- Status:.*|- Status: Approved|" "${SOURCE_DOC}" echo "> Moving ${SOURCE_DOC} to ${TARGET_DOC}..." git mv "${SOURCE_DOC}" "${TARGET_DOC}" diff --git a/text/0001-pack-suggest-stacks.md b/text/0001-pack-suggest-stacks.md index b54f041b9..d4942c8ba 100644 --- a/text/0001-pack-suggest-stacks.md +++ b/text/0001-pack-suggest-stacks.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Suggest Stacks - Start Date: 2019-04-25 +- Status: Approved - CNB Pull Request: [rfcs#4](https://github.com/buildpacks/rfcs/pull/4), [pack#190](https://github.com/buildpacks/pack/pull/190) - CNB Issue: [pack#156](https://github.com/buildpacks/pack/issues/156) - Supersedes: N/A diff --git a/text/0002-pack-logging-refactor.md b/text/0002-pack-logging-refactor.md index c7763fa81..10d1fbc2c 100644 --- a/text/0002-pack-logging-refactor.md +++ b/text/0002-pack-logging-refactor.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Pack Logging Interface - Start Date: 2019-05-06 +- Status: Approved - CNB Pull Request: [rfcs#6](https://github.com/buildpacks/rfcs/pull/6), [pack#182](https://github.com/buildpacks/pack/pull/182) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0003-pack-inspect-image.md b/text/0003-pack-inspect-image.md index f46d0ecc2..80430a77f 100644 --- a/text/0003-pack-inspect-image.md +++ b/text/0003-pack-inspect-image.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Inspect Image - Start Date: 2019-04-25 +- Status: Approved - CNB Pull Request: [rfcs#5](https://github.com/buildpacks/rfcs/pull/5) - CNB Issue: - Supersedes: N/A diff --git a/text/0004-rfc-process.md b/text/0004-rfc-process.md index 0dfc88dab..091387c49 100644 --- a/text/0004-rfc-process.md +++ b/text/0004-rfc-process.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: RFC Process - Start Date: 2019-04-09 +- Status: Superseded - CNB Pull Request: [rfcs#1](https://github.com/buildpacks/rfcs/pull/1), [rfcs#7](https://github.com/buildpacks/rfcs/pull/7) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0005-contractual-build-plan.md b/text/0005-contractual-build-plan.md index 6ac3e456a..a6679bee6 100644 --- a/text/0005-contractual-build-plan.md +++ b/text/0005-contractual-build-plan.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Contractual Build Plan - Start Date: 2019-04-12 +- Status: Approved - CNB Pull Requests: [rfcs#12](https://github.com/buildpacks/rfcs/pull/12), [spec#52](https://github.com/buildpacks/spec/pull/52), [lifecycle#149](https://github.com/buildpacks/lifecycle/pull/149) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0006-stage-specific-mixins.md b/text/0006-stage-specific-mixins.md index 543ce0ec1..e58ccecd4 100644 --- a/text/0006-stage-specific-mixins.md +++ b/text/0006-stage-specific-mixins.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Stage-specific Mixins - Start Date: 2019-06-13 +- Status: Approved - CNB Pull Requests: [rfcs#13](https://github.com/buildpacks/rfcs/pull/13), [spec#54](https://github.com/buildpacks/spec/pull/54) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0007-spec-distribution.md b/text/0007-spec-distribution.md index 41b066862..2d6118e91 100644 --- a/text/0007-spec-distribution.md +++ b/text/0007-spec-distribution.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Buildpack Distribution Specification - Start Date: 2019-04-12 +- Status: Approved - CNB Pull Requests: [rfcs#11](https://github.com/buildpacks/rfcs/pull/11), [spec#53](https://github.com/buildpacks/spec/pull/53), [lifecycle#149](https://github.com/buildpacks/lifecycle/pull/149), [pack#243](https://github.com/buildpacks/pack/issues/243) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0008-buildpack-config-for-dist.md b/text/0008-buildpack-config-for-dist.md index 2f4349a24..7612ecc83 100644 --- a/text/0008-buildpack-config-for-dist.md +++ b/text/0008-buildpack-config-for-dist.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Buildpack Configuration for Distribution - Start Date: July 22, 2019 +- Status: Approved - CNB Pull Request: [rfcs#15](https://github.com/buildpacks/rfcs/pull/15) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0009-auto-load-user-provided-environment-variables.md b/text/0009-auto-load-user-provided-environment-variables.md index cd97a24d3..5b9d86ff6 100644 --- a/text/0009-auto-load-user-provided-environment-variables.md +++ b/text/0009-auto-load-user-provided-environment-variables.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Auto-load User-provided Environment Variables - Start Date: 2019-06-17 +- Status: Approved - CNB Pull Requests: [rfcs#14](https://github.com/buildpacks/rfcs/pull/14), [spec#55](https://github.com/buildpacks/spec/pull/55), [lifecycle#163](https://github.com/buildpacks/lifecycle/pull/163) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0010-api-versions.md b/text/0010-api-versions.md index f5d571692..ad0c0d537 100644 --- a/text/0010-api-versions.md +++ b/text/0010-api-versions.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: API Versions - Start Date: 2019-08-05 +- Status: Approved - CNB Pull Request: [rfcs#19](https://github.com/buildpacks/rfcs/pull/19), [pack#282](https://github.com/buildpacks/pack/pull/282) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0011-lifecycle-descriptor.md b/text/0011-lifecycle-descriptor.md index d2b02744a..5eb4fa3ae 100644 --- a/text/0011-lifecycle-descriptor.md +++ b/text/0011-lifecycle-descriptor.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Lifecycle Descriptor - Start Date: 08/05/2019 +- Status: Superseded - CNB Pull Request: [rfcs#20](https://github.com/buildpacks/rfcs/pull/20), [lifecycle#167](https://github.com/buildpacks/lifecycle/pull/167), [pack#269](https://github.com/buildpacks/pack/pull/269), [pack#294](https://github.com/buildpacks/pack/pull/294) - CNB Issue: [pack#293](https://github.com/buildpacks/pack/issues/293) - Supersedes: N/A diff --git a/text/0012-service-binding.md b/text/0012-service-binding.md index 8bbb3e385..ce3c22d3e 100644 --- a/text/0012-service-binding.md +++ b/text/0012-service-binding.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Service Binding - Start Date: 2019-08-06 +- Status: Approved - CNB Pull Request: [rfcs#22](https://github.com/buildpacks/rfcs/pull/22), [spec#57](https://github.com/buildpacks/spec/pull/57) - CNB Issue: - Supersedes: N/A diff --git a/text/0013-app-layer-metadata-source.md b/text/0013-app-layer-metadata-source.md index d683a3bd3..cd8d7ee85 100644 --- a/text/0013-app-layer-metadata-source.md +++ b/text/0013-app-layer-metadata-source.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Manifest - App metadata - add source type, version and metadata - Start Date: 2019-06-10 +- Status: Approved - CNB Pull Request: [rfcs#9](https://github.com/buildpacks/rfcs/pull/9) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0014-combined-restorer-analyzer-phases.md b/text/0014-combined-restorer-analyzer-phases.md index a133e98f8..1c48f5384 100644 --- a/text/0014-combined-restorer-analyzer-phases.md +++ b/text/0014-combined-restorer-analyzer-phases.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Lifecycle cache contract changes. - Start Date: 2019-08-02 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0015-windows-lifecycle.md b/text/0015-windows-lifecycle.md index 65b744288..0177f44f2 100644 --- a/text/0015-windows-lifecycle.md +++ b/text/0015-windows-lifecycle.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Support for Windows in the Lifecycle components - Start Date: 2019-10-07 +- Status: Approved - CNB Pull Request: [rfcs#27](https://github.com/buildpacks/rfcs/pull/27) - CNB Issue: - Supersedes: N/A diff --git a/text/0016-use-email-more.md b/text/0016-use-email-more.md index 224ef3c1c..eb3940403 100644 --- a/text/0016-use-email-more.md +++ b/text/0016-use-email-more.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Use Email More - Start Date: 2019-12-05 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0017-pack-build-default-process-flag.md b/text/0017-pack-build-default-process-flag.md index 2363d69bd..cb98f614e 100644 --- a/text/0017-pack-build-default-process-flag.md +++ b/text/0017-pack-build-default-process-flag.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Pack Build Default Process Flag - Start Date: 2019-10-29 +- Status: Approved - CNB Pull Request: [rfcs#28](https://github.com/buildpack/rfcs/pull/28) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0018-remove-pack-run.md b/text/0018-remove-pack-run.md index ca4992736..c250fb986 100644 --- a/text/0018-remove-pack-run.md +++ b/text/0018-remove-pack-run.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Remove `pack run` - Start Date: 2019-12-11 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0019-project-descriptor.md b/text/0019-project-descriptor.md index 280f54488..8f05b534c 100644 --- a/text/0019-project-descriptor.md +++ b/text/0019-project-descriptor.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Minimal Project Descriptor - Start Date: 2019-06-11 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: https://github.com/buildpack/rfcs/pull/25 diff --git a/text/0020-landing-page.md b/text/0020-landing-page.md index 4bdff1606..b0575319b 100644 --- a/text/0020-landing-page.md +++ b/text/0020-landing-page.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Create a community landing page - Start Date: 1-6-2020 +- Status: Approved - CNB Pull Request: [rfcs#42](https://github.com/buildpack/rfcs/pull/42) - CNB Issue: 42 - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0021-lifecycle-compat-verification.md b/text/0021-lifecycle-compat-verification.md index 417d502ab..6e16fe253 100644 --- a/text/0021-lifecycle-compat-verification.md +++ b/text/0021-lifecycle-compat-verification.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Lifecycle Compatibility Verification - Start Date: 2019-12-06 +- Status: Approved - CNB Pull Request: [rfcs#34](https://github.com/buildpacks/rfcs/pull/34) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0022-client-side-buildpack-registry.md b/text/0022-client-side-buildpack-registry.md index 76f69bc60..d8d52cdaf 100644 --- a/text/0022-client-side-buildpack-registry.md +++ b/text/0022-client-side-buildpack-registry.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Client-Side Buildpack Registry - Start Date: 2019-11-21 +- Status: Superseded - CNB Pull Request: [rfcs#35](https://github.com/buildpacks/rfcs/pull/35) - CNB Issue: (leave blank) - Supersedes: https://github.com/buildpack/rfcs/pull/24 diff --git a/text/0023-circleci-orb.md b/text/0023-circleci-orb.md index 4600fe5d1..92b529abe 100644 --- a/text/0023-circleci-orb.md +++ b/text/0023-circleci-orb.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Support a CircleCI Orb - Start Date: 2019-12-15 +- Status: Approved - CNB Pull Request: [rfcs#39](https://github.com/buildpacks/rfcs/pull/39) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0024-lifecycle-multicall-binary-for-build.md b/text/0024-lifecycle-multicall-binary-for-build.md index 0b22a1fd1..355af9ded 100644 --- a/text/0024-lifecycle-multicall-binary-for-build.md +++ b/text/0024-lifecycle-multicall-binary-for-build.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Lifecycle as a multicall binary for build phases - Start Date: 01/21/2020 +- Status: Approved - CNB Pull Request: [rfcs#45](https://github.com/buildpacks/rfcs/pull/45) - CNB Issue: (leave blank) - Supersedes: "N/A" diff --git a/text/0025-dont-trust-builders.md b/text/0025-dont-trust-builders.md index 5089fbc81..925965a47 100644 --- a/text/0025-dont-trust-builders.md +++ b/text/0025-dont-trust-builders.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Isolate Registry Credentials from Builder Images - Start Date: 2020-01-14 +- Status: Approved - CNB Pull Request: [rfcs#43](https://github.com/buildpacks/rfcs/pull/43) - CNB Issue: - Supersedes: N/A diff --git a/text/0026-lifecycle-all.md b/text/0026-lifecycle-all.md index 279924051..c24ab0c0c 100644 --- a/text/0026-lifecycle-all.md +++ b/text/0026-lifecycle-all.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Lifecycle All Binary - Start Date: 2020-01-21 +- Status: Approved - CNB Pull Request: [rfcs#46](https://github.com/buildpacks/rfcs/pull/46) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0027-spec-api-branches.md b/text/0027-spec-api-branches.md index 351e3780c..5b320b638 100644 --- a/text/0027-spec-api-branches.md +++ b/text/0027-spec-api-branches.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Spec API branches - Start Date: 2020-01-27 +- Status: Approved - CNB Pull Request: [rfcs#49](https://github.com/buildpacks/rfcs/pull/49) - CNB Issue: - Supersedes: N/A diff --git a/text/0028-remove-cred-helpers.md b/text/0028-remove-cred-helpers.md index 32eb14686..001cd624b 100644 --- a/text/0028-remove-cred-helpers.md +++ b/text/0028-remove-cred-helpers.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Remove Lifecycle Credential Helpers Integration - Start Date: 2020-01-22 +- Status: Approved - CNB Pull Request: [rfcs#49](https://github.com/buildpacks/rfcs/pull/49) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0029-template-changes.md b/text/0029-template-changes.md index faef401f7..cf245ef59 100644 --- a/text/0029-template-changes.md +++ b/text/0029-template-changes.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: RFC template changes - Start Date: 2020-02-03 +- Status: Approved - CNB Pull Request: [rfcs#52](https://github.com/buildpacks/rfcs/pull/52) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0030-links-for-buildpacks.md b/text/0030-links-for-buildpacks.md index 42dc369fe..7774fb2aa 100644 --- a/text/0030-links-for-buildpacks.md +++ b/text/0030-links-for-buildpacks.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Buildpack homepage - Start Date: 1-23-2020 +- Status: Approved - CNB Pull Request: [rfcs#55](https://github.com/buildpacks/rfcs/pull/55) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0031-bionic-mixins.md b/text/0031-bionic-mixins.md index 159a88e95..9952fa8f7 100644 --- a/text/0031-bionic-mixins.md +++ b/text/0031-bionic-mixins.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Defining Mixins for io.buildpacks.stacks.bionic - Start Date: 2019-12-05 +- Status: Approved - CNB Pull Request: https://github.com/buildpacks/rfcs/pull/40 - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0032-update-json-cnb-registry.md b/text/0032-update-json-cnb-registry.md index 9acba4140..72aaf2086 100644 --- a/text/0032-update-json-cnb-registry.md +++ b/text/0032-update-json-cnb-registry.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Update JSON Structure of CNB Registry - Start Date: 2020-03-19 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: [RFC-0022](https://github.com/buildpacks/rfcs/blob/main/text/0022-client-side-buildpack-registry.md) diff --git a/text/0033-add-author.md b/text/0033-add-author.md index 6fe705670..54ef37ba7 100644 --- a/text/0033-add-author.md +++ b/text/0033-add-author.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Add Author to RFC Metadata - Start Date: 2020-03-14 +- Status: Approved - CNB Pull Request: https://github.com/buildpacks/rfcs/pull/64 - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0034-image-labels.md b/text/0034-image-labels.md index 968964637..91e7788de 100644 --- a/text/0034-image-labels.md +++ b/text/0034-image-labels.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Image Labels - Start Date: 2020-03-24 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0035-buildpack-versions.md b/text/0035-buildpack-versions.md index 8a0085deb..98024a3a0 100644 --- a/text/0035-buildpack-versions.md +++ b/text/0035-buildpack-versions.md @@ -2,6 +2,7 @@ [meta]: #meta - **Name:** Semantic Version Formats for Buildpack Versions - **Start Date:** 2020-03-11 +- **Status:** Approved - **Supersedes:** N/A # Summary diff --git a/text/0036-cnb-buildpack-directory-env-var.md b/text/0036-cnb-buildpack-directory-env-var.md index 51ba4625f..1b0b78d92 100644 --- a/text/0036-cnb-buildpack-directory-env-var.md +++ b/text/0036-cnb-buildpack-directory-env-var.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: CNB\_BUILDPACK\_DIR environment variable - Start Date: 2020-04-02 +- Status: Approved - CNB Pull Request: [rfcs#49](https://github.com/buildpacks/rfcs/pull/71) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0037-buildpack-uris.md b/text/0037-buildpack-uris.md index 92776495a..616c759fd 100644 --- a/text/0037-buildpack-uris.md +++ b/text/0037-buildpack-uris.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Buildpack URIs - Start Date: 02/07/2020 +- Status: Approved - CNB Pull Request: [rfcs#56](https://github.com/buildpacks/rfcs/pull/56) - CNB Issue: (leave blank) - Supersedes: "N/A" diff --git a/text/0038-gh-repo-labels.md b/text/0038-gh-repo-labels.md index f4bfdd840..5fe9160a4 100644 --- a/text/0038-gh-repo-labels.md +++ b/text/0038-gh-repo-labels.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: GitHub repo labels - Start Date: 02/03/2020 +- Status: Approved - CNB Pull Request: [rfcs#53](https://github.com/buildpacks/rfcs/pull/53) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0039-release-process.md b/text/0039-release-process.md index 3333a02fd..931a643e6 100644 --- a/text/0039-release-process.md +++ b/text/0039-release-process.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Release Process - Start Date: 2019-11-18 +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0040-export-report.md b/text/0040-export-report.md index 9b4a06b7c..d87c87a76 100644 --- a/text/0040-export-report.md +++ b/text/0040-export-report.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Export Report - Start Date: (fill in today's date: 2020-04-01) +- Status: Approved - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: "N/A" diff --git a/text/0041-api-version-compat.md b/text/0041-api-version-compat.md index 5d5ff70c3..ad00d6a98 100644 --- a/text/0041-api-version-compat.md +++ b/text/0041-api-version-compat.md @@ -3,6 +3,7 @@ - Name: Lifecycle API version support changes - Start Date: 2020-05-14 - Author(s): [Emily Casey](https://github.com/ekcasey) +- Status: Approved - RFC Pull Request: [rfcs#79](https://github.com/buildpacks/rfcs/pull/79) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0042-process-specific-env.md b/text/0042-process-specific-env.md index a455fc2fc..297a71276 100644 --- a/text/0042-process-specific-env.md +++ b/text/0042-process-specific-env.md @@ -2,6 +2,7 @@ [meta]: #meta - Name: Process Specific Environment - Start Date: 2020-04-03 +- Status: Approved - CNB Pull Request: (https://github.com/buildpacks/rfcs/pull/72) - CNB Issue: - Supersedes: N/A diff --git a/text/0043-increase-build-plan-flexibility.md b/text/0043-increase-build-plan-flexibility.md index d31670963..eb51238b0 100644 --- a/text/0043-increase-build-plan-flexibility.md +++ b/text/0043-increase-build-plan-flexibility.md @@ -3,6 +3,7 @@ - Name: Increase Build Plan Flexibility - Start Date: 2020-05-28 - Author(s): nebhale +- Status: Approved - RFC Pull Request: [rfcs#82](https://github.com/buildpacks/rfcs/pull/82) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0044-pack-publish-buildpack.md b/text/0044-pack-publish-buildpack.md index 183b1f607..dc0c8ec53 100644 --- a/text/0044-pack-publish-buildpack.md +++ b/text/0044-pack-publish-buildpack.md @@ -3,6 +3,7 @@ - Name: Pack Register Buildpack - Start Date: 2020-04-27 - Author(s): [Joe Kutner](https://github.com/jkutner), [Javier Romero](https://github.com/jromero) +- Status: Approved - RFC Pull Request: [rfcs#75](https://github.com/buildpacks/rfcs/pull/75) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0045-launcher-arguments.md b/text/0045-launcher-arguments.md index 77f6acfba..5a5bbf85d 100644 --- a/text/0045-launcher-arguments.md +++ b/text/0045-launcher-arguments.md @@ -3,6 +3,7 @@ - Name: Launcher Arguments - Start Date: 2020-06-02 - Author(s): nebhale ekcasey +- Status: Superseded - RFC Pull Request: [rfcs#84](https://github.com/buildpacks/rfcs/pull/84) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0046-pack-pull-policy.md b/text/0046-pack-pull-policy.md index 5276e1f06..f6e7bd9d9 100644 --- a/text/0046-pack-pull-policy.md +++ b/text/0046-pack-pull-policy.md @@ -3,6 +3,7 @@ - Name: `pack` Image Pull Policy - Start Date: 2020-05-21 - Author(s): [Javier Romero](https://github.com/jromero) +- Status: Approved - RFC Pull Request: [rfcs#80](https://github.com/buildpacks/rfcs/pull/80) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0047-danger-zone.md b/text/0047-danger-zone.md index 60515b137..5188ede4a 100644 --- a/text/0047-danger-zone.md +++ b/text/0047-danger-zone.md @@ -3,6 +3,7 @@ - Name: Read-Write Volume Mount in Pack - Start Date: 2020-06-02 - Author(s): nebhale +- Status: Approved - RFC Pull Request: [rfcs#85](https://github.com/buildpacks/rfcs/pull/85) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0048-inline-buildpack.md b/text/0048-inline-buildpack.md index 9785a4d21..b8557d183 100644 --- a/text/0048-inline-buildpack.md +++ b/text/0048-inline-buildpack.md @@ -3,6 +3,7 @@ - Name: Inline Buildpacks - Start Date: 2020-06-11 - Author(s): [Joe Kutner](https://github.com/jkutner) +- Status: Approved - RFC Pull Request: [rfcs#86](https://github.com/buildpacks/rfcs/pull/86) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0049-multi-api-lifecycle-descriptor.md b/text/0049-multi-api-lifecycle-descriptor.md index 9ba9427f9..93763a172 100644 --- a/text/0049-multi-api-lifecycle-descriptor.md +++ b/text/0049-multi-api-lifecycle-descriptor.md @@ -3,6 +3,7 @@ - Name: Lifecycle Descriptor with Multiple APIs - Start Date: 2020-07-02 - Author(s): ekcasey +- Status: Superseded - RFC Pull Request: [rfcs#92](https://github.com/buildpacks/rfcs/pull/92) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0050-stack-metadata.md b/text/0050-stack-metadata.md index b3bcaa41c..3adbe8bed 100644 --- a/text/0050-stack-metadata.md +++ b/text/0050-stack-metadata.md @@ -3,6 +3,7 @@ - Name: Additional Stack Metadata - Start Date: 2020-05-12 - Author(s): kvedurmu +- Status: Approved - RFC Pull Request: [rfcs#78](https://github.com/buildpacks/rfcs/pull/78) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0051-override-env-by-default.md b/text/0051-override-env-by-default.md index 6fb5b7ffa..82d22eded 100644 --- a/text/0051-override-env-by-default.md +++ b/text/0051-override-env-by-default.md @@ -3,6 +3,7 @@ - Name: Override Env Vars by Default - Start Date: 2020-07-26 - Author(s): @sclevine +- Status: Approved - RFC Pull Request: [rfcs#98](https://github.com/buildpacks/rfcs/pull/98) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0052-opt-in-layer-caching.md b/text/0052-opt-in-layer-caching.md index 32daa2eb6..b54763649 100644 --- a/text/0052-opt-in-layer-caching.md +++ b/text/0052-opt-in-layer-caching.md @@ -3,6 +3,7 @@ - Name: Opt-in Layer Caching - Start Date: 2020-07-26 - Author(s): @sclevine +- Status: Approved - RFC Pull Request: [rfcs#99](https://github.com/buildpacks/rfcs/pull/99) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0053-decouple-buildpack-plan-and-bom.md b/text/0053-decouple-buildpack-plan-and-bom.md index dc6cc2d4c..44231d8bf 100644 --- a/text/0053-decouple-buildpack-plan-and-bom.md +++ b/text/0053-decouple-buildpack-plan-and-bom.md @@ -3,6 +3,7 @@ - Name: Decouple Buildpack Plan and Bill-of-Materials - Start Date: 2020-07-26 - Author(s): @sclevine +- Status: Superseded - RFC Pull Request: [rfcs#100](https://github.com/buildpacks/rfcs/pull/100) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0054-project-descriptor-schema.md b/text/0054-project-descriptor-schema.md index ac066f5c8..a8d094769 100644 --- a/text/0054-project-descriptor-schema.md +++ b/text/0054-project-descriptor-schema.md @@ -3,6 +3,7 @@ - Name: Project Descriptor (project.toml) Schema - Start Date: 2020-28-07 - Author(s): @joshwlewis +- Status: Approved - RFC Pull Request: - CNB Pull Request: - CNB Issue: diff --git a/text/0055-deprecate-service-bindings.md b/text/0055-deprecate-service-bindings.md index c36a91177..d9be209a8 100644 --- a/text/0055-deprecate-service-bindings.md +++ b/text/0055-deprecate-service-bindings.md @@ -3,6 +3,7 @@ - Name: Deprecate Bindings Extension Specification - Start Date: 2020-08-06 - Author(s): @nebhale +- Status: Approved - RFC Pull Request: [rfcs#105](https://github.com/buildpacks/rfcs/pull/105) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0056-any-stack-buildpacks.md b/text/0056-any-stack-buildpacks.md index 15d7fd9e2..67b2eab53 100644 --- a/text/0056-any-stack-buildpacks.md +++ b/text/0056-any-stack-buildpacks.md @@ -3,6 +3,7 @@ - Name: Stackless Buildpacks - Start Date: 2020-07-26 - Author(s): @sclevine +- Status: Approved - RFC Pull Request: [rfcs#97](https://github.com/buildpacks/rfcs/pull/97) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0057-exec.d-shell-free-profile-d.md b/text/0057-exec.d-shell-free-profile-d.md index 58e77f1de..b250c27bc 100644 --- a/text/0057-exec.d-shell-free-profile-d.md +++ b/text/0057-exec.d-shell-free-profile-d.md @@ -3,6 +3,7 @@ - Name: Exec.d - Shell-Free Profile.d - Start Date: 2020-07-26 - Author(s): @sclevine +- Status: Approved - RFC Pull Request: [rfcs#104](https://github.com/buildpacks/rfcs/pull/104) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0058-pack-subcommands.md b/text/0058-pack-subcommands.md index ce9ef9f16..95544d32c 100644 --- a/text/0058-pack-subcommands.md +++ b/text/0058-pack-subcommands.md @@ -3,6 +3,7 @@ - Name: `pack` subcommands - Start Date: 2020-07-08 - Author(s): @jromero +- Status: Approved - RFC Pull Request: [rfcs#93](https://github.com/buildpacks/rfcs/pull/93) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0059-label-rfcs.md b/text/0059-label-rfcs.md index 0cdea9ddf..8c70203cf 100644 --- a/text/0059-label-rfcs.md +++ b/text/0059-label-rfcs.md @@ -3,6 +3,7 @@ - Name: Label RFCs With Specification And Target Audience - Start Date: 2020-08-15 - Author(s): ForestEckhardt +- Status: Approved - RFC Pull Request: [rfcs#108](https://github.com/buildpacks/rfcs/pull/108) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0060-create-repo-issues.md b/text/0060-create-repo-issues.md index ef0e03d9e..4791f2b10 100644 --- a/text/0060-create-repo-issues.md +++ b/text/0060-create-repo-issues.md @@ -3,6 +3,7 @@ - Name: Create Repo Issues - Start Date: 2020-08-06 - Author(s): [@dfreilich][https://github.com/dfreilich] +- Status: Approved - RFC Pull Request: [rfcs#106](https://github.com/buildpacks/rfcs/pull/106) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0061-relax-mixin-contract.md b/text/0061-relax-mixin-contract.md index 3481abbb7..463913a56 100644 --- a/text/0061-relax-mixin-contract.md +++ b/text/0061-relax-mixin-contract.md @@ -3,6 +3,7 @@ - Name: Relax Mixin Contract - Start Date: 2020-08-12 - Author(s): @sclevine +- Status: Approved - RFC Pull Request: [rfcs#109](https://github.com/buildpacks/rfcs/pull/109) - CNB Pull Request: (leave blank) - CNB Issue: https://github.com/buildpacks/spec/issues/149, https://github.com/buildpacks/pack/issues/868, https://github.com/buildpacks/lifecycle/issues/425 diff --git a/text/0062-distribution-team.md b/text/0062-distribution-team.md index 68c219985..0043b4d5e 100644 --- a/text/0062-distribution-team.md +++ b/text/0062-distribution-team.md @@ -3,6 +3,7 @@ - Name: Create a Distribution Team - Start Date: 2020-09-03 - Author(s): [@jkutner](@jkutner) +- Status: Approved - RFC Pull Request: [rfcs#113](https://github.com/buildpacks/rfcs/pull/113) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/community#35](https://github.com/buildpacks/community/issues/35) diff --git a/text/0063-stack-images-config-env-path-must-exist.md b/text/0063-stack-images-config-env-path-must-exist.md index 4582cabc4..abbbf84b0 100644 --- a/text/0063-stack-images-config-env-path-must-exist.md +++ b/text/0063-stack-images-config-env-path-must-exist.md @@ -3,6 +3,7 @@ - Name: Build/Run image configs MUST contain env.PATH - Start Date: 2020-08-21 - Author(s): @aemengo @ameyer-pivotal @mvalliath @TisVictress @micahyoung +- Status: Approved - RFC Pull Request: [rfcs#114](https://github.com/buildpacks/rfcs/pull/114) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#150](https://github.com/buildpacks/spec/issues/150), [buildpacks/docs#223](https://github.com/buildpacks/docs/issues/223) diff --git a/text/0064-buildpacks-contribute-default-process-type.md b/text/0064-buildpacks-contribute-default-process-type.md index d42e7688b..3c28a145e 100644 --- a/text/0064-buildpacks-contribute-default-process-type.md +++ b/text/0064-buildpacks-contribute-default-process-type.md @@ -3,6 +3,7 @@ - Name: Buildpacks should be able to define the default process type for an app - Start Date: 8/20/20 - Author(s): natalieparellano +- Status: Approved - RFC Pull Request: [rfcs#110](https://github.com/buildpacks/rfcs/pull/110) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#159](https://github.com/buildpacks/spec/issues/159), [buildpacks/spec#160](https://github.com/buildpacks/spec/issues/160), [buildpacks/lifecycle#457](https://github.com/buildpacks/lifecycle/issues/457), [buildpacks/lifecycle#458](https://github.com/buildpacks/lifecycle/issues/458), [buildpacks/pack#946](https://github.com/buildpacks/pack/issues/946), [buildpacks/docs#246](https://github.com/buildpacks/docs/issues/246) diff --git a/text/0065-builder-spec.md b/text/0065-builder-spec.md index 15602b33f..dec3a283b 100644 --- a/text/0065-builder-spec.md +++ b/text/0065-builder-spec.md @@ -3,6 +3,7 @@ - Name: Builder Spec - Start Date: 2020-09-11 - Author(s): [@dfreilich](https://github.com/dfreilich) +- Status: Approved - RFC Pull Request: [rfcs#116](https://github.com/buildpacks/rfcs/pull/116) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#101](https://github.com/buildpacks/spec/issues/101), [buildpacks/pack#945](https://github.com/buildpacks/pack/issues/945) diff --git a/text/0066-lifecycle-prelease-version-and-experimental-section.md b/text/0066-lifecycle-prelease-version-and-experimental-section.md index bd503f1b3..201e353c1 100644 --- a/text/0066-lifecycle-prelease-version-and-experimental-section.md +++ b/text/0066-lifecycle-prelease-version-and-experimental-section.md @@ -3,6 +3,7 @@ - Name: Prelease APIs and Experimental Features - Start Date: 2020-08-25 - Author(s): @hone +- Status: Approved - RFC Pull Request: [rfcs#115](https://github.com/buildpacks/rfcs/pull/115) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#161](https://github.com/buildpacks/spec/issues/161), [buildpacks/lifecycle#459](https://github.com/buildpacks/lifecycle/issues/459), [buildpacks/lifecycle#460](https://github.com/buildpacks/lifecycle/issues/460), [buildpacks/pack#947](https://github.com/buildpacks/pack/issues/947) diff --git a/text/0067-report-toml-image-manifest-size.md b/text/0067-report-toml-image-manifest-size.md index 3ddf0929d..24f7cfe6c 100644 --- a/text/0067-report-toml-image-manifest-size.md +++ b/text/0067-report-toml-image-manifest-size.md @@ -3,6 +3,7 @@ - Name: Add Image Manifest Size to report.toml - Start Date: 2020-11-02 - Author(s): djoyahoy +- Status: Approved - RFC Pull Request: [rfcs#121](https://github.com/buildpacks/rfcs/pull/121) - CNB Pull Request: - CNB Issue: [buildpacks/spec#169](https://github.com/buildpacks/spec/issues/169), [buildpacks/lifecycle#490](https://github.com/buildpacks/lifecycle/issues/490) diff --git a/text/0068-buildpack-registry-search-api.md b/text/0068-buildpack-registry-search-api.md index edce91d72..40e97b6eb 100644 --- a/text/0068-buildpack-registry-search-api.md +++ b/text/0068-buildpack-registry-search-api.md @@ -3,6 +3,7 @@ - Name: Buildpack Registry Search API - Start Date: 2020-11-11 - Author(s): @elbandito +- Status: Approved - RFC Pull Request: [rfcs#125](https://github.com/buildpacks/rfcs/pull/125) - CNB Pull Request: (leave blank) - CNB Issue: [registry-api#3](https://github.com/buildpacks/registry-api/issues/3) diff --git a/text/0069-stack-buildpacks.md b/text/0069-stack-buildpacks.md index bf1c89e8c..fb2d28adb 100644 --- a/text/0069-stack-buildpacks.md +++ b/text/0069-stack-buildpacks.md @@ -3,6 +3,7 @@ - Name: Stack Buildpacks - Start Date: 2020-08-27 - Author(s): [@jkutner](@jkutner) +- Status: Superseded - RFC Pull Request: [rfcs#111](https://github.com/buildpacks/rfcs/pull/111) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#177](https://github.com/buildpacks/spec/issues/177), [buildpacks/spec#178](https://github.com/buildpacks/spec/issues/178), [buildpacks/spec#170](https://github.com/buildpacks/spec/issues/170), [buildpacks/lifecycle#501](https://github.com/buildpacks/lifecycle/issues/501), [buildpacks/pack#1021](https://github.com/buildpacks/pack/issues/1021) diff --git a/text/0070-new-buildpack-toml-keys.md b/text/0070-new-buildpack-toml-keys.md index b5f40f82a..5661433d9 100644 --- a/text/0070-new-buildpack-toml-keys.md +++ b/text/0070-new-buildpack-toml-keys.md @@ -3,6 +3,7 @@ - Name: New Buildpack Descriptor Keys - Start Date: 2020-12-12 - Author(s): [@jkutner](@jkutner) +- Status: Approved - RFC Pull Request: [rfcs#127](https://github.com/buildpacks/rfcs/pull/127) - CNB Pull Request: [buildpacks/pack#1022](https://github.com/buildpacks/pack/pull/1022) - CNB Issue: [buildpacks/spec#181](https://github.com/buildpacks/spec/issues/181), [buildpacks/docs#288](https://github.com/buildpacks/docs/issues/288) diff --git a/text/0071-cnb-user-research-2021.md b/text/0071-cnb-user-research-2021.md index ebd4a7c8a..bc9013a33 100644 --- a/text/0071-cnb-user-research-2021.md +++ b/text/0071-cnb-user-research-2021.md @@ -3,6 +3,7 @@ - Name: CNB User Research 2021 - Start Date: 2020-12-10 - Author(s): [@sampeinado](@sampeinado) +- Status: Approved - RFC Pull Request: [rfcs#126](https://github.com/buildpacks/rfcs/pull/126) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0072-image-workdir.md b/text/0072-image-workdir.md index 20d265333..2a15c136e 100644 --- a/text/0072-image-workdir.md +++ b/text/0072-image-workdir.md @@ -3,6 +3,7 @@ - Name: Set Image Working Directory to value of CNB_APP_DIR - Start Date: 2021-01-13 - Author(s): @josegonzalez +- Status: Approved - RFC Pull Request: [rfcs#134](https://github.com/buildpacks/rfcs/pull/134) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#183](https://github.com/buildpacks/spec/issues/183), [buildpacks/lifecycle#516](https://github.com/buildpacks/lifecycle/issues/516), [buildpacks/docs#290](https://github.com/buildpacks/docs/issues/290) diff --git a/text/0073-offline-buildpackages.md b/text/0073-offline-buildpackages.md index 66eb9995b..b875fc907 100644 --- a/text/0073-offline-buildpackages.md +++ b/text/0073-offline-buildpackages.md @@ -3,11 +3,16 @@ - Name: Asset Cache - Start Date: 2020-04-27 - Author(s): dwillist +- Status: On Hold - RFC Pull Request: [rfcs#81](https://github.com/buildpacks/rfcs/pull/81) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#184](https://github.com/buildpacks/spec/issues/184), [buildpacks/pack#1055](https://github.com/buildpacks/pack/issues/1055), [buildpacks/lifecycle#521](https://github.com/buildpacks/lifecycle/issues/521) - Supersedes: NA +# Notes +## On Hold +Since the approval of this RFC, additional concerns around multiple caching strategies within CNB has been raised. Solving this problem is still a priority for the project but the core team has decided to place this RFC on hold pending further discussion of a more holistic caching solution. If you are interested in this problem, please participate in any of our channels. + # Summary [summary]: #summary diff --git a/text/0074-layer-table.md b/text/0074-layer-table.md index 4f5873a32..14d9bfc26 100644 --- a/text/0074-layer-table.md +++ b/text/0074-layer-table.md @@ -3,6 +3,7 @@ - Name: Move layer types to new table in layer.toml - Start Date: 2021-01-05 - Author(s): natalieparellano +- Status: Approved - RFC Pull Request: [rfcs#132](https://github.com/buildpacks/rfcs/pull/132) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#185](https://github.com/buildpacks/spec/issues/185), [buildpacks/lifecycle#522](https://github.com/buildpacks/lifecycle/issues/522), [buildpacks/docs#296](https://github.com/buildpacks/docs/issues/296) diff --git a/text/0075-move-analyze-phase.md b/text/0075-move-analyze-phase.md index 03a4ba9a0..ee87bdc5e 100644 --- a/text/0075-move-analyze-phase.md +++ b/text/0075-move-analyze-phase.md @@ -3,6 +3,7 @@ - Name: Move analyze phase - Start Date: 2021-01-13 - Author(s): [@jkutner](github.com/jkutner/) [@jabrown85](github.com/jabrown85) +- Status: Approved - RFC Pull Request: [rfcs#135](https://github.com/buildpacks/rfcs/pull/135) - CNB Pull Request: [buildpacks/spec#172](https://github.com/buildpacks/spec/pull/172) - CNB Issue: [buildpacks/spec#194](https://github.com/buildpacks/spec/issues/194), [buildpacks/lifecycle#530](https://github.com/buildpacks/lifecycle/issues/530), [buildpacks/pack#1078](https://github.com/buildpacks/pack/issues/1078), [buildpacks/docs#302](https://github.com/buildpacks/docs/issues/302) diff --git a/text/0076-windows-security-identifiers.md b/text/0076-windows-security-identifiers.md index d3bc19827..c939be4ad 100644 --- a/text/0076-windows-security-identifiers.md +++ b/text/0076-windows-security-identifiers.md @@ -3,6 +3,7 @@ - Name: Use Security Identifiers for Windows User/Group IDs - Start Date: 2021-01-25 - Author(s): micahyoung +- Status: Approved - RFC Pull Request: [rfcs#133](https://github.com/buildpacks/rfcs/pull/133) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#129](https://github.com/buildpacks/spec/issues/129), [buildpacks/lifecycle#343](https://github.com/buildpacks/lifecycle/issues/343), [buildpacks/pack#1079](https://github.com/buildpacks/pack/issues/1079), [buildpacks/docs#303](https://github.com/buildpacks/docs/issues/303) diff --git a/text/0077-pack-buildpack-create.md b/text/0077-pack-buildpack-create.md index f422dd764..850ea1485 100644 --- a/text/0077-pack-buildpack-create.md +++ b/text/0077-pack-buildpack-create.md @@ -3,6 +3,7 @@ - Name: Pack Command to Create a Buildpack Repo - Start Date: 2021-01-19 - Author(s): [jkutner](https://github.com/jkutner) +- Status: Approved - RFC Pull Request: [rfcs#136](https://github.com/buildpacks/rfcs/pull/136) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1025](https://github.com/buildpacks/pack/issues/1025) diff --git a/text/0078-group-additions.md b/text/0078-group-additions.md index 91ffc7685..9eed25872 100644 --- a/text/0078-group-additions.md +++ b/text/0078-group-additions.md @@ -3,6 +3,7 @@ - Name: Group additions to Builder order - Start Date: 2020-12-23 - Author(s): [jkutner](@jkutner) +- Status: Approved - RFC Pull Request: [rfcs#129](https://github.com/buildpacks/rfcs/pull/129) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/docs#319](https://github.com/buildpacks/docs/issues/319), [buildpacks/pack#1099](https://github.com/buildpacks/pack/issues/1099), [buildpacks/pack#1100](https://github.com/buildpacks/pack/issues/1100), [buildpacks/spec#195](https://github.com/buildpacks/spec/issues/195) diff --git a/text/0079-create-stackify-repo.md b/text/0079-create-stackify-repo.md index d3d113cae..6c8079efc 100644 --- a/text/0079-create-stackify-repo.md +++ b/text/0079-create-stackify-repo.md @@ -3,6 +3,7 @@ - Name: Create stackify repo - Start Date: 2020-11-06 - Authors: @martyspiewak @mdelillo @dumez-k +- Status: Approved - RFC Pull Request: [rfcs#123](https://github.com/buildpacks/rfcs/pull/123) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1101](https://github.com/buildpacks/pack/issues/1101), [buildpacks/docs#321](https://github.com/buildpacks/docs/issues/321) diff --git a/text/0080-builder-key-in-project-descriptor.md b/text/0080-builder-key-in-project-descriptor.md index b71cc6c76..293842ef9 100644 --- a/text/0080-builder-key-in-project-descriptor.md +++ b/text/0080-builder-key-in-project-descriptor.md @@ -3,6 +3,7 @@ - Name: Builder key in project descriptor - Start Date: 2021-02-11 - Author(s): @wburningham +- Status: Approved - RFC Pull Request: [rfcs#138](https://github.com/buildpacks/rfcs/pull/138) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#215](https://github.com/buildpacks/spec/issues/215), [buildpacks/pack#1114](https://github.com/buildpacks/pack/issues/1114), [buildpacks/docs#345](https://github.com/buildpacks/docs/issues/345) diff --git a/text/0081-process-specific-working-directory.md b/text/0081-process-specific-working-directory.md index b22fca9ab..370df0e1d 100644 --- a/text/0081-process-specific-working-directory.md +++ b/text/0081-process-specific-working-directory.md @@ -3,6 +3,7 @@ - Name: Process Specific Working Directory - Start Date: 2021-03-09 - Author(s): ForestEckhardt, ryanmoran, fg-j +- Status: Approved - RFC Pull Request: [rfcs#144](https://github.com/buildpacks/rfcs/pull/144) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#212](https://github.com/buildpacks/spec/issues/212), [buildpacks/spec#216](https://github.com/buildpacks/spec/issues/216) diff --git a/text/0082-pack-package-refactor.md b/text/0082-pack-package-refactor.md index 871c6e8b1..54bc382a9 100644 --- a/text/0082-pack-package-refactor.md +++ b/text/0082-pack-package-refactor.md @@ -3,6 +3,7 @@ - Name: Refactor Pack go packages - Start Date: 02/24/21 - Author(s): dwillist +- Status: Approved - RFC Pull Request: [rfcs#139](https://github.com/buildpacks/rfcs/pull/139) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1128](https://github.com/buildpacks/pack/issues/1128) diff --git a/text/0083-best-practices-and-guidelines.md b/text/0083-best-practices-and-guidelines.md index 6a2f1d283..cb87bec23 100644 --- a/text/0083-best-practices-and-guidelines.md +++ b/text/0083-best-practices-and-guidelines.md @@ -3,6 +3,7 @@ - Name: Best practices and guidelines for Cloud Native Buildpacks - Start Date: 2021-03-29 - Author(s): [@samj1912](https://github.com/samj1912) +- Status: Approved - RFC Pull Request: [rfcs#150](https://github.com/buildpacks/rfcs/pull/150) - CNB Pull Request: (leave blank) - CNB Issue: N/A diff --git a/text/0084-project-descriptor-domains.md b/text/0084-project-descriptor-domains.md index 3de4815ed..bb3077fe0 100644 --- a/text/0084-project-descriptor-domains.md +++ b/text/0084-project-descriptor-domains.md @@ -3,6 +3,7 @@ - Name: Project Descriptor Domains - Start Date: 2021-02-26 - Author(s): hone +- Status: Approved - RFC Pull Request: [rfcs#140](https://github.com/buildpacks/rfcs/pull/140) - CNB Pull Request: (leave blank) - CNB Issue: N/A diff --git a/text/0084-rfc-issue-generation.md b/text/0084-rfc-issue-generation.md index 6a95cde61..8d112829f 100644 --- a/text/0084-rfc-issue-generation.md +++ b/text/0084-rfc-issue-generation.md @@ -3,6 +3,7 @@ - Name: RFC Issue Generation - Start Date: 2020-03-03 - Author(s): @jromero +- Status: Approved - RFC Pull Request: [rfcs#141](https://github.com/buildpacks/rfcs/pull/141) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/rfcs#158](https://github.com/buildpacks/rfcs/issues/158) diff --git a/text/0085-run-uid.md b/text/0085-run-uid.md index 0de420daf..dd6c89060 100644 --- a/text/0085-run-uid.md +++ b/text/0085-run-uid.md @@ -3,6 +3,7 @@ - Name: Recommending different run-time and build-time users - Start Date: 2021-03-15 - Author(s): [@samj1912](https://github.com/samj1912) +- Status: Approved - RFC Pull Request: [rfcs#146](https://github.com/buildpacks/rfcs/pull/146) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#223](https://github.com/buildpacks/spec/issues/223) diff --git a/text/0086-component-level-contrib.md b/text/0086-component-level-contrib.md index 64e7e6bca..fe88fe3f1 100644 --- a/text/0086-component-level-contrib.md +++ b/text/0086-component-level-contrib.md @@ -3,6 +3,7 @@ - Name: Guidelines for Component-level Contributions - Start Date: 2021-03-08 - Author(s): @sclevine +- Status: Approved - RFC Pull Request: [rfcs#143](https://github.com/buildpacks/rfcs/pull/143) - CNB Pull Request: (leave blank) - CNB Issue: [community#93](https://github.com/buildpacks/community/issues/93) diff --git a/text/0087-bom-in-layer-metadata.md b/text/0087-bom-in-layer-metadata.md index 8beb05d70..aa0f294e7 100644 --- a/text/0087-bom-in-layer-metadata.md +++ b/text/0087-bom-in-layer-metadata.md @@ -3,6 +3,7 @@ - Name: BOM inclusion in layer content metadata - Start Date: 2021-04-01 - Author(s): [@samj1912](https://github.com/samj1912) +- Status: Superseded - RFC Pull Request: [rfcs#151](https://github.com/buildpacks/rfcs/pull/151) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#227](https://github.com/buildpacks/spec/issues/227), [buildpacks/lifecycle#623](https://github.com/buildpacks/lifecycle/issues/623), [buildpacks/docs#375](https://github.com/buildpacks/docs/issues/375) diff --git a/text/0088-minimum-docs-lifecycle.md b/text/0088-minimum-docs-lifecycle.md index 1fbd1811d..d07603b36 100644 --- a/text/0088-minimum-docs-lifecycle.md +++ b/text/0088-minimum-docs-lifecycle.md @@ -3,6 +3,7 @@ - Name: Define a minimum standard for docs in order to ship the lifecycle - Start Date: 4/7/2021 - Author(s): natalieparellano +- Status: Approved - RFC Pull Request: [rfcs#153](https://github.com/buildpacks/rfcs/pull/153) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/lifecycle#627](https://github.com/buildpacks/lifecycle/issues/627) diff --git a/text/0089-buildpack-authors-tooling-subteam.md b/text/0089-buildpack-authors-tooling-subteam.md index 9128faf0b..61da5ed0c 100644 --- a/text/0089-buildpack-authors-tooling-subteam.md +++ b/text/0089-buildpack-authors-tooling-subteam.md @@ -3,6 +3,7 @@ - Name: Buildpack Authors' Tooling Sub-Team - Start Date: 2021-05-11 - Author(s): [@ekcasey](https://github.com/ekcasey), [@hone](https://github.com/hone), [@samj1912](https://github.com/samj1912) +- Status: Approved - RFC Pull Request: [rfcs#159](https://github.com/buildpacks/rfcs/pull/159) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/community#102](https://github.com/buildpacks/community/issues/102) diff --git a/text/0090-intro-video-script.md b/text/0090-intro-video-script.md index 2358bb031..8ebe32947 100644 --- a/text/0090-intro-video-script.md +++ b/text/0090-intro-video-script.md @@ -3,6 +3,7 @@ - Name: Intro video script - Start Date: 2021-05-18 - Author(s): yaelharel +- Status: Approved - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0091-pack-cache-options.md b/text/0091-pack-cache-options.md index 37e3d6f4a..d4cdf2aaf 100644 --- a/text/0091-pack-cache-options.md +++ b/text/0091-pack-cache-options.md @@ -3,6 +3,7 @@ - Name: Pack cache options - Start Date: 2021-03-25 - Author(s): [@jromero](https://github.com/jromero), [@dwillist](https://github.com/dwillist) +- Status: Approved - RFC Pull Request: [rfcs#149](https://github.com/buildpacks/rfcs/pull/149) - CNB Pull Request: (leave blank) - CNB Issue: [pack#1077](https://github.com/buildpacks/pack/issues/1077) diff --git a/text/0092-add-visual-pack-build.md b/text/0092-add-visual-pack-build.md index 7d1d271f8..b20e16657 100644 --- a/text/0092-add-visual-pack-build.md +++ b/text/0092-add-visual-pack-build.md @@ -3,6 +3,7 @@ - Name: Add Visual Pack Build - Start Date: 2020-05-14 - Author(s): aemengo +- Status: Approved - RFC Pull Request: [rfcs#160](https://github.com/buildpacks/rfcs/pull/160) - CNB Pull Request: (leave blank) - CNB Issue: [pack#1200](https://github.com/buildpacks/pack/issues/1200), [pack#1201](https://github.com/buildpacks/pack/issues/1201), [pack#1203](https://github.com/buildpacks/pack/issues/1203), [pack#1204](https://github.com/buildpacks/pack/issues/1204), [pack#1205](https://github.com/buildpacks/pack/issues/1205), [pack#1206](https://github.com/buildpacks/pack/issues/1206) diff --git a/text/0093-remove-shell-processes.md b/text/0093-remove-shell-processes.md index ab54f63f7..0e9a88d50 100644 --- a/text/0093-remove-shell-processes.md +++ b/text/0093-remove-shell-processes.md @@ -3,6 +3,7 @@ - Name: Remove Shell Processes - Start Date: 2021-05-14 - Author(s): @ekcasey +- Status: Approved - RFC Pull Request: [rfcs#168](https://github.com/buildpacks/rfcs/pull/168) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#244](https://github.com/buildpacks/spec/issues/244), [buildpacks/spec#245](https://github.com/buildpacks/spec/issues/245), [buildpacks/lifecycle#693](https://github.com/buildpacks/lifecycle/issues/693), [buildpacks/pack#1260](https://github.com/buildpacks/pack/issues/1260), [buildpacks/docs#391](https://github.com/buildpacks/docs/issues/391), [buildpacks/libcnb#70](https://github.com/buildpacks/libcnb/issues/70) diff --git a/text/0094-add-status-field.md b/text/0094-add-status-field.md index be154b9aa..5dd946abb 100644 --- a/text/0094-add-status-field.md +++ b/text/0094-add-status-field.md @@ -3,7 +3,7 @@ - Name: Add Status to RFC Metadata - Start Date: 2021-07-14 - Author(s): @aemengo, @ekcasey -- Status: Draft Approved +- Status: Approved - RFC Pull Request: [rfcs#177](https://github.com/buildpacks/rfcs/pull/177) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/rfcs#183](#https://github.com/buildpacks/rfcs/issues/183) diff --git a/text/0095-sbom.md b/text/0095-sbom.md index c96f43c17..7c7ae04be 100644 --- a/text/0095-sbom.md +++ b/text/0095-sbom.md @@ -3,6 +3,7 @@ - Name: Structured SBOMs - Start Date: 2021-06-02 - Author(s): [@samj1912](https://github.com/samj1912), [@sophiewigmore](https://github.com/sophiewigmore), [@ForestEckhardt](https://github.com/ForestEckhardt) +- Status: Approved - RFC Pull Request: [rfcs#166](https://github.com/buildpacks/rfcs/pull/166) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/lifecycle#732](https://github.com/buildpacks/lifecycle/issues/732), [buildpacks/lifecycle#733](https://github.com/buildpacks/lifecycle/issues/733), [buildpacks/lifecycle#734](https://github.com/buildpacks/lifecycle/issues/734), [buildpacks/lifecycle#735](https://github.com/buildpacks/lifecycle/issues/735), [buildpacks/lifecycle#736](https://github.com/buildpacks/lifecycle/issues/736), [buildpacks/lifecycle#737](https://github.com/buildpacks/lifecycle/issues/737), [buildpacks/lifecycle#738](https://github.com/buildpacks/lifecycle/issues/738) diff --git a/text/0096-remove-stacks-mixins.md b/text/0096-remove-stacks-mixins.md index f2acc4238..31dee5d68 100644 --- a/text/0096-remove-stacks-mixins.md +++ b/text/0096-remove-stacks-mixins.md @@ -3,6 +3,7 @@ - Name: Remove Stacks & Mixins - Start Date: 2021-06-30 - Author(s): sclevine +- Status: Approved - RFC Pull Request: [rfcs#172](https://github.com/buildpacks/rfcs/pull/172) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1301](https://github.com/buildpacks/pack/issues/1301), [buildpacks/spec#258](https://github.com/buildpacks/spec/issues/258), [buildpacks/samples#111](https://github.com/buildpacks/samples/issues/111), [buildpacks/docs#422](https://github.com/buildpacks/docs/issues/422), [buildpacks/spec#259](https://github.com/buildpacks/spec/issues/259), [buildpacks/spec#260](https://github.com/buildpacks/spec/issues/260), [buildpacks/lifecycle#742](https://github.com/buildpacks/lifecycle/issues/742), [buildpacks/lifecycle#743](https://github.com/buildpacks/lifecycle/issues/743), [buildpacks/lifecycle#744](https://github.com/buildpacks/lifecycle/issues/744), [buildpacks/pack#1302](https://github.com/buildpacks/pack/issues/1302), [buildpacks/pack#1303](https://github.com/buildpacks/pack/issues/1303), [buildpacks/pack#1304](https://github.com/buildpacks/pack/issues/1304) From defcefa722aca339af60bb8e6a6c2b346a7a67d5 Mon Sep 17 00:00:00 2001 From: Anthony Emengo Date: Mon, 17 Jan 2022 14:08:42 -0500 Subject: [PATCH 025/372] Address PR reviews Signed-off-by: Anthony Emengo --- text/0061-relax-mixin-contract.md | 2 +- text/0079-create-stackify-repo.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0061-relax-mixin-contract.md b/text/0061-relax-mixin-contract.md index 463913a56..8cdd1635f 100644 --- a/text/0061-relax-mixin-contract.md +++ b/text/0061-relax-mixin-contract.md @@ -3,7 +3,7 @@ - Name: Relax Mixin Contract - Start Date: 2020-08-12 - Author(s): @sclevine -- Status: Approved +- Status: Superseded - RFC Pull Request: [rfcs#109](https://github.com/buildpacks/rfcs/pull/109) - CNB Pull Request: (leave blank) - CNB Issue: https://github.com/buildpacks/spec/issues/149, https://github.com/buildpacks/pack/issues/868, https://github.com/buildpacks/lifecycle/issues/425 diff --git a/text/0079-create-stackify-repo.md b/text/0079-create-stackify-repo.md index 6c8079efc..2c73ab470 100644 --- a/text/0079-create-stackify-repo.md +++ b/text/0079-create-stackify-repo.md @@ -3,7 +3,7 @@ - Name: Create stackify repo - Start Date: 2020-11-06 - Authors: @martyspiewak @mdelillo @dumez-k -- Status: Approved +- Status: Superseded - RFC Pull Request: [rfcs#123](https://github.com/buildpacks/rfcs/pull/123) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1101](https://github.com/buildpacks/pack/issues/1101), [buildpacks/docs#321](https://github.com/buildpacks/docs/issues/321) From 9eb6251271c3a0d6f5fa085c6a73e72c48beb504 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 27 Jan 2022 09:52:02 -0600 Subject: [PATCH 026/372] Update text/0000-buildpack-input-vars.md Signed-off-by: Joe Kutner --- text/0000-buildpack-input-vars.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md index ef3e7d1e4..1cea47483 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0000-buildpack-input-vars.md @@ -39,7 +39,7 @@ We will deprecate the positional arguments to `bin/detect` and `bin/build` with ### bin/detect * `CNB_PLATFORM_DIR` - replaces the first positional argument to `bin/build`. Uses the same env var name as the Platform spec. -* `CNB_PLAN_PATH` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. +* `CNB_BUILD_PLAN_PATH` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. ### bin/build From 335f5112ab9e2f404a53cd01ed531bbfc75e8088 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Tue, 7 Dec 2021 13:39:03 +0000 Subject: [PATCH 027/372] Add proposal to move to CNCF slack Signed-off-by: Sambhav Kothari --- text/0000-slack.md | 105 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 text/0000-slack.md diff --git a/text/0000-slack.md b/text/0000-slack.md new file mode 100644 index 000000000..6c0daf688 --- /dev/null +++ b/text/0000-slack.md @@ -0,0 +1,105 @@ +# Meta +[meta]: #meta +- Name: Migrate to CNCF Slack +- Start Date: 2021-12-07 +- Author(s): [@samj1912](https://github.com/samj1912) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +This RFC proposes that we move our independent slack instance to the CNCF slack instance. + +# Definitions +[definitions]: #definitions + +N/A + +# Motivation +[motivation]: #motivation + +CNCF slack is an umbrella slack enterprise instance that quite a few CNCF projects use. As Buildpacks is maturing as a CNCF project, using the CNCF slack will put us closer to our sibling projects and their community. The CNCF slack also comes with CNCF support around spam prevention, moderation and integration with LFX insights. This will help us manage and track the growth of our community. CNCF slack is also an enterprise instance which means that we will have unlimited retention of messages and the ability to create people groups to mention maintainer teams. We will also be able to use private channels if needed. + +As a part of CNCF slack onboarding, we can also migrate all of our data and channels onto the CNCF workspace, preserving any past history of conversations that took place in the Buildpacks slack. For more details please see [CNCF - Migrating workspaces](https://slack.com/intl/en-gb/help/articles/217872578-Import-data-from-one-Slack-workspace-to-another) + +# What it is +[what-it-is]: #what-it-is + +This RFC proposes that we move the Buildpacks slack instance from an independent instance to the CNCF instance. + +# How it Works +[how-it-works]: #how-it-works + +The maintainers will have to request the migration of the Buildpacks slack over to CNCF slack and co-ordinate the announcements/user migration. + +The following are the slack channels that will be migrated over and their proposed names after migrations - + +| Current | Proposed | +| ----------------------- | ------------------------------ | +| announcements | buildpacks-announcements | +| arm | buildpacks-arch-arm | +| buildpacks-authors | buildpacks-authors-team | +| buildpacks-distribution | buildpacks-distribution | +| general | buildpacks | +| implementation | buildpacks-implementation-team | +| implementation-ops | buildpacks-implementation-ops | +| learning | buildpacks-learning-team | +| learning-ops | buildpacks-learning-ops | +| maintainers | buildpacks-maintainers | +| mentoring | buildpacks-mentoring | +| ops | buildpacks-ci-ops | +| pack-cli | buildpacks-pack-cli | +| platform | buildpacks-platform-team | +| registry | buildpacks-registry | +| spec | buildpacks-spec | +| spec-ops | buildpacks-spec-ops | +| tekton | buildpacks-platform-tekton | +| windows | buildpacks-arch-windows | + +The following channels will not be migrated - + +- classic (inactive) +- community (inactive) +- delivery (inactive) +- docs (inactive) +- elixir (inactive) +- governance (inactive) +- java (inactive) +- kaniko (inactive) +- kpack (not part of the official project) +- packfile (inactive) +- python (inactive) +- random (inactive) +- rfcs (inactive) +- ruby (inactive) +- stacks (inactive) +- summit-2021 (inactive) +- user-research (inactive) +- wg-chat (inactive) + + +# Drawbacks +[drawbacks]: #drawbacks + +- All of the current users are not moved over to the new slack instance as a part of the migration. Users who are in the Buildpacks slack but not in the CNCF slack will have to create new accounts on the CNCF slack. +- The migration might also be disruptive to the existing Buildpacks community on slack, as such we should carefully evaluate whether we want to move or not. + +# Alternatives +[alternatives]: #alternatives + +- Keep using the Buildpacks slack. +- Use the Kubernetes slack instance, which is the other alternative that CNCF provides. Kubernetes slack has a larger user base but doesn't come with the ability to create private channels. + +# Prior Art +[prior-art]: #prior-art + +TBD + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +How to handle the migration with no downtime? From 30f660bb83a216995b75bc3f119e4722c97d14c4 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Tue, 1 Feb 2022 21:35:59 +0000 Subject: [PATCH 028/372] Update text/0000-slack.md Signed-off-by: Sambhav Kothari Co-authored-by: Terence Lee Signed-off-by: Sambhav Kothari --- text/0000-slack.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/text/0000-slack.md b/text/0000-slack.md index 6c0daf688..ccad5b159 100644 --- a/text/0000-slack.md +++ b/text/0000-slack.md @@ -80,7 +80,25 @@ The following channels will not be migrated - - summit-2021 (inactive) - user-research (inactive) - wg-chat (inactive) - +## Migration Steps +* [ ] once RFC finalizes, announce migration in slack and mailing list with move over date and a 1 hour maintenance window where chat will be down during the migration. +* [ ] before Feb 2, get slack backup of history +* [ ] on date move over + * [ ] setup final announcement in slack and set all channels as read-only. + * [ ] backup history merge with previous backup, give to CNCF for migration + * [ ] merge docs PR pointing to the new slack. + +### Backups +Slack backups are a zip file that have this layout: +``` +/YYYY-MM-DD.json +... +channels.json +integration_logs.json +users.json +``` + +The plan is to take an initial dump on Feb 1 of all the history up to that point. The second backup will contain the history from Jan 31 up to the the cut off time. The Feb 1->cut off dates will be copied into the original zip and the `channels.json`, `integration_logs.json`, and `users.json` will be copied over as well into the root. # Drawbacks [drawbacks]: #drawbacks From 3611d65f9a38056bb50606a5cfed539ff9f966e1 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 08:04:36 -0600 Subject: [PATCH 029/372] rename CNB_LAYERS_DIR in platform spec Signed-off-by: Joe Kutner --- text/0000-buildpack-input-vars.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md index 1cea47483..ced4672cc 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0000-buildpack-input-vars.md @@ -43,7 +43,7 @@ We will deprecate the positional arguments to `bin/detect` and `bin/build` with ### bin/build -* `CNB_BP_LAYERS_DIR` - replaces the first positional argument to `bin/build`. Uses `_BP_` to differentiate it from the `CNB_LAYERS_DIR` in the Plaform spec, which is a different value. +* `CNB_LAYERS_DIR` - replaces the first positional argument to `bin/build`. **Note:** `CNB_LAYERS_DIR` conflicts with the platform spec, which will beupdated to rename its `CNB_LAYERS_DIR` to `CNB_PARENT_LAYERS_DIR`. * `CNB_PLATFORM_DIR` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. * `CNB_PLAN_PATH` - replaces the third positional argument to `bin/build`. Uses the same env var name as the Platform spec. @@ -81,7 +81,11 @@ N/A # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes -In the Buildpack spec: +## Platform Spec + +Rename `CNB_LAYERS_DIR` to `CNB_PARENT_LAYERS_DIR`. + +## Buildpack spec ### Detection From 81b700b50c6044f75a7683d002e683a6060ee1e0 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 28 Jul 2021 22:23:06 -0500 Subject: [PATCH 030/372] Add RFC for System Buildpacks Signed-off-by: Joe Kutner --- text/0000-system-buildpacks-in-builder.md | 155 ++++++++++++++++++++++ 1 file changed, 155 insertions(+) create mode 100644 text/0000-system-buildpacks-in-builder.md diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md new file mode 100644 index 000000000..868334485 --- /dev/null +++ b/text/0000-system-buildpacks-in-builder.md @@ -0,0 +1,155 @@ +# Meta +[meta]: #meta +- Name: System Buildpacks in Builder Images +- Start Date: 2021-07-24 +- Author(s): [@jkutner](https://github.com/jkutner) +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This is a proposal for a mechanism that would allow a builder to contain a default set of buildpacks that run on every build, independent of which buildpack group passes detection. + +# Definitions +[definitions]: #definitions + +* _system buildpacks_ - a standard buildpack, conforming to the Buildpack API, but which runs outside of normal groups + +# Motivation +[motivation]: #motivation + +Forthcoming changes to the lifecycle (such as [removal of shell-specific logic](https://github.com/buildpacks/rfcs/pull/168)) will remove capabilities that users have come to expect. This includes mechanisms like `.profile`, which allows a buildpack user to customize the environment a process type runs in. We seek to replace these lost mechanisms with buildpacks, in an effort to preserve the capability while still removing complexity from the lifecycle. + +# What it is +[what-it-is]: #what-it-is + +We introduce a `[system]` table in the `builder.toml` schema with the following structure: + +``` +[[system.pre.buildpacks]] + id = "" + version = "" + optional = false + +[[system.post.buildpacks]] + id = "" + version = "" + optional = false +``` + +The fields in the `system.pre.buildpacks` table and `system.post.buildpacks` table match the fields in the existing [`order.group` table](https://buildpacks.io/docs/reference/config/builder-config/#order-_list-required_). + +When a builder includes one or more `system.*.buildpacks` entry, each build phase that uses the builder will run the `pre` buildpacks before ALL other buildpacks (whether they are defined by a default `order.group` or if they are provided as an explicit order to the lifecycle) and the `post` buildpack after ALL other buildpacks. + +# How it Works +[how-it-works]: #how-it-works + +Unless otherwise stated, system buildpacks conform to the [buildpack API](https://github.com/buildpacks/spec/blob/main/buildpack.md). + +The `system.pre.buildpacks` will be transformed into a new table in `order.toml`. The `[system]` table in `order.toml` will be processed by the lifecycle, and each `pre`/`post` buildpack will run during the detect phase. Those that pass detection will run during the build phase. + +## Detection + +The results of detection by system buildpacks MUST NOT influence the selected buildpack group. If no system buildpacks pass detection, any buildpack group MAY pass detection. If a system buildpack pass detection and no buildpack groups pass detection, then detection MUST fail. + +System buildpacks may require/provide in the build plan following standard buildpack API specification. + +A new flag to the lifecycle `detector`, `--disable-system-buildpacks`, will disable system buildpacks + +## Build + +System buildpacks that have passed detection will be executed during the build phase. All `pre` buildpacks must execute before the detected buildpack group. All `post` buildpacks must execute after the detected buildpack group. However, the system buildpacks MUST NOT be added to the buildpack group, and their execution may be hidden from the end user. + +If a system buildpack exits with a status of `100`, the build will fail. + +A new flag to the lifecycle `builder`, `--disable-system-buildpacks`, will disable system buildpacks + +# Drawbacks +[drawbacks]: #drawbacks + +- Default buildpacks are essentially hidden from the user and may be unexpected. + +# Alternatives +[alternatives]: #alternatives + +## Do Nothing + +End users would have to add buildpacks like the `profile-buildpack` or other buildpacks that implement system/spec behaviors themselves. + +## Use the `[[order]]` table + +Instead of a new `[system]` table, we could put `pre` and `post` in the `[[order]]` table. However, this could imply that there is a interaction/override/etc between these buildpacks and the `pre`/`post` buildpacks in `project.toml`. But there is not. + +## Use the `[lifecycle]` table + +``` +[lifecycle] + version = "" + + [[lifecycle.pre.buildpacks]] + id = "" + version = "" + optional = false + + [[lifecycle.post.buildpacks]] + id = "" + version = "" + optional = false +``` + +# Prior Art +[prior-art]: #prior-art + +- [RFC-0078: Group additions to Builder order](https://github.com/buildpacks/rfcs/blob/main/text/0078-group-additions.md) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- Should the system buildpacks that pass detection be added to the buildpack group? This might make it difficult to hide them. +- Should the system buildpacks output to stdout/stderr and should that be displayed? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +## `detector` in Platform specifiction + +This proposal introduces a `--disable-system-buildpacks` flag on the `detector`. + +``` +/cnb/lifecycle/detector \ + [--disable-system-buildpacks] \ +``` + +## `builder` in Platform specifiction + +This proposal introduces a `--disable-system-buildpacks` flag on the `builder`. + +``` +/cnb/lifecycle/builder \ + [--disable-system-buildpacks] \ +``` + +## `[[order.toml]]` in Platform specifiction + +This proposal requires changes to the [`order.toml` schema](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml). + +``` +[[system.pre.buildpacks]] + id = "" + version = "" + optional = false + +[[system.post.buildpacks]] + id = "" + version = "" + optional = false +``` + +Where: +* Both `id` and `version` MUST be present for each buildpack object in a group. +* The value of `optional` MUST default to false if not specified. + + From e130502b92c568f0c686523b5cf092a041ef535e Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 3 Dec 2021 07:32:53 -0600 Subject: [PATCH 031/372] Clarified verbiage for system buildpacks Signed-off-by: Joe Kutner --- text/0000-system-buildpacks-in-builder.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 868334485..c9a4fed55 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -53,11 +53,11 @@ The `system.pre.buildpacks` will be transformed into a new table in `order.toml` ## Detection -The results of detection by system buildpacks MUST NOT influence the selected buildpack group. If no system buildpacks pass detection, any buildpack group MAY pass detection. If a system buildpack pass detection and no buildpack groups pass detection, then detection MUST fail. +The exit code of detection by system buildpacks MUST NOT influence the selected buildpack group. If no system buildpacks pass detection, any buildpack group MAY pass detection. If a system buildpack passes detection and no buildpack groups pass detection, then detection MUST fail. System buildpacks may require/provide in the build plan following standard buildpack API specification. -A new flag to the lifecycle `detector`, `--disable-system-buildpacks`, will disable system buildpacks +A new flag to the lifecycle `detector`, `--disable-system-buildpacks`, will disable system buildpacks. ## Build @@ -65,12 +65,12 @@ System buildpacks that have passed detection will be executed during the build p If a system buildpack exits with a status of `100`, the build will fail. -A new flag to the lifecycle `builder`, `--disable-system-buildpacks`, will disable system buildpacks +A new flag to the lifecycle `builder`, `--disable-system-buildpacks`, will disable system buildpacks. # Drawbacks [drawbacks]: #drawbacks -- Default buildpacks are essentially hidden from the user and may be unexpected. +- Default system buildpacks are hidden from the user before the build and their execution may be unexpected. # Alternatives [alternatives]: #alternatives From 1eaee804ff0f0636b16febd15c4376c756d8231c Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 2 Feb 2022 14:20:03 -0600 Subject: [PATCH 032/372] Updates based on WG feedback Signed-off-by: Joe Kutner --- text/0000-system-buildpacks-in-builder.md | 43 ++++++----------------- 1 file changed, 11 insertions(+), 32 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index c9a4fed55..18454346a 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -32,15 +32,15 @@ We introduce a `[system]` table in the `builder.toml` schema with the following [[system.pre.buildpacks]] id = "" version = "" - optional = false + optional = true [[system.post.buildpacks]] id = "" version = "" - optional = false + optional = true ``` -The fields in the `system.pre.buildpacks` table and `system.post.buildpacks` table match the fields in the existing [`order.group` table](https://buildpacks.io/docs/reference/config/builder-config/#order-_list-required_). +The fields in the `system.pre.buildpacks` table and `system.post.buildpacks` table match the fields in the existing [`order.group` table](https://buildpacks.io/docs/reference/config/builder-config/#order-_list-required_). However, `optional` in this case is required and the only acceptable value is `true`. Non-optional buildpacks will cause the builder creation to fail. When a builder includes one or more `system.*.buildpacks` entry, each build phase that uses the builder will run the `pre` buildpacks before ALL other buildpacks (whether they are defined by a default `order.group` or if they are provided as an explicit order to the lifecycle) and the `post` buildpack after ALL other buildpacks. @@ -109,47 +109,26 @@ Instead of a new `[system]` table, we could put `pre` and `post` in the `[[order [unresolved-questions]: #unresolved-questions - Should the system buildpacks that pass detection be added to the buildpack group? This might make it difficult to hide them. + - disposition: yes. hiding is optional and up to the platform - Should the system buildpacks output to stdout/stderr and should that be displayed? + - disposition: it may be # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes ## `detector` in Platform specifiction -This proposal introduces a `--disable-system-buildpacks` flag on the `detector`. +This proposal introduces a `--pre-buildpacks` and `--post-buildpacks` option on the `detector`. ``` /cnb/lifecycle/detector \ - [--disable-system-buildpacks] \ + [--pre-buildpacks ]\ + [--post-buildpacks ]\ ``` -## `builder` in Platform specifiction +The lifecycle: -This proposal introduces a `--disable-system-buildpacks` flag on the `builder`. - -``` -/cnb/lifecycle/builder \ - [--disable-system-buildpacks] \ -``` - -## `[[order.toml]]` in Platform specifiction - -This proposal requires changes to the [`order.toml` schema](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml). - -``` -[[system.pre.buildpacks]] - id = "" - version = "" - optional = false - -[[system.post.buildpacks]] - id = "" - version = "" - optional = false -``` - -Where: -* Both `id` and `version` MUST be present for each buildpack object in a group. -* The value of `optional` MUST default to false if not specified. +* SHALL merge the `` group with each group from `` such that the `pre` buildpacks are placed at the beginning of each group before running detection. +* SHALL merge the `` group with each group from `` such that the `post` buildpacks are placed at the end of each group before running detection. From 87c01bab70c8bc83b2cb1af843acb8c1a8823b46 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 3 Feb 2022 08:59:03 -0600 Subject: [PATCH 033/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 18454346a..fcc57be81 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -11,7 +11,7 @@ # Summary [summary]: #summary -This is a proposal for a mechanism that would allow a builder to contain a default set of buildpacks that run on every build, independent of which buildpack group passes detection. +This is a proposal for a mechanism that would allow a builder to contain a default set of buildpacks that participate in every detection group, regardless of the buildpack order passed by the platform. # Definitions [definitions]: #definitions From d77ef97bf084d5c9587e66c66255db576701d89d Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 3 Feb 2022 08:59:16 -0600 Subject: [PATCH 034/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index fcc57be81..3cddc3429 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -16,7 +16,7 @@ This is a proposal for a mechanism that would allow a builder to contain a defau # Definitions [definitions]: #definitions -* _system buildpacks_ - a standard buildpack, conforming to the Buildpack API, but which runs outside of normal groups +* _system buildpacks_ - a standard buildpack, conforming to the Buildpack API, which participate in all groups # Motivation [motivation]: #motivation From daed18e7ed777ebe13d3ccfe58809f60a5fc9c69 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 3 Feb 2022 09:00:52 -0600 Subject: [PATCH 035/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 3cddc3429..6fb5e6ff9 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -42,7 +42,7 @@ We introduce a `[system]` table in the `builder.toml` schema with the following The fields in the `system.pre.buildpacks` table and `system.post.buildpacks` table match the fields in the existing [`order.group` table](https://buildpacks.io/docs/reference/config/builder-config/#order-_list-required_). However, `optional` in this case is required and the only acceptable value is `true`. Non-optional buildpacks will cause the builder creation to fail. -When a builder includes one or more `system.*.buildpacks` entry, each build phase that uses the builder will run the `pre` buildpacks before ALL other buildpacks (whether they are defined by a default `order.group` or if they are provided as an explicit order to the lifecycle) and the `post` buildpack after ALL other buildpacks. +When a builder includes one or more `system.*.buildpacks` entry, the detect phase will prepend and append all `pre` and `post` buildpacks to each detection group in the provided order, respectively. # How it Works [how-it-works]: #how-it-works From 1ec097e777cf5dd57a25a800db7c6d2bdc8597c8 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 3 Feb 2022 09:04:43 -0600 Subject: [PATCH 036/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 6fb5e6ff9..7854542f3 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -61,7 +61,7 @@ A new flag to the lifecycle `detector`, `--disable-system-buildpacks`, will disa ## Build -System buildpacks that have passed detection will be executed during the build phase. All `pre` buildpacks must execute before the detected buildpack group. All `post` buildpacks must execute after the detected buildpack group. However, the system buildpacks MUST NOT be added to the buildpack group, and their execution may be hidden from the end user. +System buildpacks that have passed detection will be added to `group.toml` and treated like any other buildpack for the remainder of the build. If a system buildpack exits with a status of `100`, the build will fail. From 3d9d225579f38d318bcbc799561719d201697295 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 3 Feb 2022 09:04:57 -0600 Subject: [PATCH 037/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 7854542f3..a469cf9b3 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -65,8 +65,6 @@ System buildpacks that have passed detection will be added to `group.toml` and t If a system buildpack exits with a status of `100`, the build will fail. -A new flag to the lifecycle `builder`, `--disable-system-buildpacks`, will disable system buildpacks. - # Drawbacks [drawbacks]: #drawbacks From 51bd0d93e24c187b436ed9b82633b8ef0bedf525 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 3 Feb 2022 14:16:33 -0600 Subject: [PATCH 038/372] add system.toml for detect Signed-off-by: Joe Kutner --- text/0000-system-buildpacks-in-builder.md | 27 ++++++++++++++++------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index a469cf9b3..7cf87a833 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -49,7 +49,7 @@ When a builder includes one or more `system.*.buildpacks` entry, the detect phas Unless otherwise stated, system buildpacks conform to the [buildpack API](https://github.com/buildpacks/spec/blob/main/buildpack.md). -The `system.pre.buildpacks` will be transformed into a new table in `order.toml`. The `[system]` table in `order.toml` will be processed by the lifecycle, and each `pre`/`post` buildpack will run during the detect phase. Those that pass detection will run during the build phase. +The `system.*pre*.buildpacks` will be provided to the lifecycle into a new file, `system.toml`. The `[system]` table in `system.toml` will be processed by the lifecycle, and each `pre`/`post` buildpack will run during the detect phase. Those that pass detection will run during the build phase. ## Detection @@ -57,8 +57,6 @@ The exit code of detection by system buildpacks MUST NOT influence the selected System buildpacks may require/provide in the build plan following standard buildpack API specification. -A new flag to the lifecycle `detector`, `--disable-system-buildpacks`, will disable system buildpacks. - ## Build System buildpacks that have passed detection will be added to `group.toml` and treated like any other buildpack for the remainder of the build. @@ -120,13 +118,26 @@ This proposal introduces a `--pre-buildpacks` and `--post-buildpacks` option on ``` /cnb/lifecycle/detector \ - [--pre-buildpacks ]\ - [--post-buildpacks ]\ + [--system ]\ ``` -The lifecycle: +Where: + +* the lifecycle SHALL merge the `pre` group with each group from `` such that the `pre` buildpacks are placed at the beginning of each order group before running detection. +* SHALL merge the `post` group with each group from `` such that the `post` buildpacks are placed at the end of each order group before running detection. -* SHALL merge the `` group with each group from `` such that the `pre` buildpacks are placed at the beginning of each group before running detection. -* SHALL merge the `` group with each group from `` such that the `post` buildpacks are placed at the end of each group before running detection. +#### `system.toml` (TOML) + +```toml +[[system.pre.buildpacks]] + id = "" + version = "" + optional = true + +[[system.post.buildpacks]] + id = "" + version = "" + optional = true +``` From e7a6d9a5a1757947719048392e0b541d7b7a082b Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 08:16:31 -0600 Subject: [PATCH 039/372] udpates to allow optional system buildpacks Signed-off-by: Joe Kutner --- text/0000-system-buildpacks-in-builder.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 7cf87a833..fc1b390eb 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -32,18 +32,20 @@ We introduce a `[system]` table in the `builder.toml` schema with the following [[system.pre.buildpacks]] id = "" version = "" - optional = true + optional = false [[system.post.buildpacks]] id = "" version = "" - optional = true + optional = false ``` -The fields in the `system.pre.buildpacks` table and `system.post.buildpacks` table match the fields in the existing [`order.group` table](https://buildpacks.io/docs/reference/config/builder-config/#order-_list-required_). However, `optional` in this case is required and the only acceptable value is `true`. Non-optional buildpacks will cause the builder creation to fail. +The fields in the `system.pre.buildpacks` table and `system.post.buildpacks` table match the fields in the existing [`order.group` table](https://buildpacks.io/docs/reference/config/builder-config/#order-_list-required_). When a builder includes one or more `system.*.buildpacks` entry, the detect phase will prepend and append all `pre` and `post` buildpacks to each detection group in the provided order, respectively. +**Note:** A non-`optional` system buildpack creates the possibility that a user provided group with all optional buildpacks could pass detection when it otherwise would not. We leave that up to the platform/builder owner. As long as the platform has a mechanism to disable system buildpacks (and `pack` will), then there is an escape valve for this situation. + # How it Works [how-it-works]: #how-it-works @@ -53,7 +55,7 @@ The `system.*pre*.buildpacks` will be provided to the lifecycle into a new file, ## Detection -The exit code of detection by system buildpacks MUST NOT influence the selected buildpack group. If no system buildpacks pass detection, any buildpack group MAY pass detection. If a system buildpack passes detection and no buildpack groups pass detection, then detection MUST fail. +The exit code of detection by system buildpacks MUST influence the selected buildpack group. If a system buildpack is non-optional and fails detection, then detection MUST for that group fail. If a system buildpack is optional and passes detection, then detection MAY pass for that group. System buildpacks may require/provide in the build plan following standard buildpack API specification. From f1a4e026d2cf84a43d0ea5cb5f684c916879bba7 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Fri, 4 Feb 2022 10:43:16 -0600 Subject: [PATCH 040/372] Update text/0000-buildpack-input-vars.md Signed-off-by: Terence Lee --- text/0000-buildpack-input-vars.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md index ced4672cc..ad258388d 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0000-buildpack-input-vars.md @@ -43,7 +43,7 @@ We will deprecate the positional arguments to `bin/detect` and `bin/build` with ### bin/build -* `CNB_LAYERS_DIR` - replaces the first positional argument to `bin/build`. **Note:** `CNB_LAYERS_DIR` conflicts with the platform spec, which will beupdated to rename its `CNB_LAYERS_DIR` to `CNB_PARENT_LAYERS_DIR`. +* `CNB_LAYERS_DIR` - replaces the first positional argument to `bin/build`. **Note:** `CNB_LAYERS_DIR` conflicts with the platform spec, which will be updated to rename its `CNB_LAYERS_DIR` to `CNB_PARENT_LAYERS_DIR`. * `CNB_PLATFORM_DIR` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. * `CNB_PLAN_PATH` - replaces the third positional argument to `bin/build`. Uses the same env var name as the Platform spec. From dcb69f400a72eb32d56c01068f790524619234e5 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Fri, 4 Feb 2022 14:09:46 -0600 Subject: [PATCH 041/372] re-index the second 0084 RFC to 0098 RFC --- ...{0084-rfc-issue-generation.md => 0098-rfc-issue-generation.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0084-rfc-issue-generation.md => 0098-rfc-issue-generation.md} (100%) diff --git a/text/0084-rfc-issue-generation.md b/text/0098-rfc-issue-generation.md similarity index 100% rename from text/0084-rfc-issue-generation.md rename to text/0098-rfc-issue-generation.md From eb7217ebfbeead8a406108bac95a116c394e1bdc Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 14:39:58 -0600 Subject: [PATCH 042/372] fix spec in system buildpacks Signed-off-by: Joe Kutner --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index fc1b390eb..c2a1afeec 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -116,7 +116,7 @@ Instead of a new `[system]` table, we could put `pre` and `post` in the `[[order ## `detector` in Platform specifiction -This proposal introduces a `--pre-buildpacks` and `--post-buildpacks` option on the `detector`. +This proposal introduces a `--system` option on the `detector`. ``` /cnb/lifecycle/detector \ From e269b1325b9d2fa6c49be75533ce57095baef5b7 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 14:52:34 -0600 Subject: [PATCH 043/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index c2a1afeec..9eed87cef 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -108,8 +108,6 @@ Instead of a new `[system]` table, we could put `pre` and `post` in the `[[order - Should the system buildpacks that pass detection be added to the buildpack group? This might make it difficult to hide them. - disposition: yes. hiding is optional and up to the platform -- Should the system buildpacks output to stdout/stderr and should that be displayed? - - disposition: it may be # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From e03e627992c95a04ecc1b562560e7f0c890301a5 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 14:52:50 -0600 Subject: [PATCH 044/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 9eed87cef..94373a2fb 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -106,8 +106,6 @@ Instead of a new `[system]` table, we could put `pre` and `post` in the `[[order # Unresolved Questions [unresolved-questions]: #unresolved-questions -- Should the system buildpacks that pass detection be added to the buildpack group? This might make it difficult to hide them. - - disposition: yes. hiding is optional and up to the platform # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From 48d1da2fc2988a1a333bb50b3434d229770db5e8 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 14:53:06 -0600 Subject: [PATCH 045/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 94373a2fb..9f667dae8 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -68,7 +68,7 @@ If a system buildpack exits with a status of `100`, the build will fail. # Drawbacks [drawbacks]: #drawbacks -- Default system buildpacks are hidden from the user before the build and their execution may be unexpected. +- If system buildpacks are hidden from the user before the build, their execution may be unexpected. # Alternatives [alternatives]: #alternatives From e32acbb24389d9d3b212495aff5ff1b42f4df189 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 14:53:20 -0600 Subject: [PATCH 046/372] Update text/0000-system-buildpacks-in-builder.md Signed-off-by: Joe Kutner Co-authored-by: Emily Casey --- text/0000-system-buildpacks-in-builder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0000-system-buildpacks-in-builder.md index 9f667dae8..645a14299 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0000-system-buildpacks-in-builder.md @@ -49,7 +49,7 @@ When a builder includes one or more `system.*.buildpacks` entry, the detect phas # How it Works [how-it-works]: #how-it-works -Unless otherwise stated, system buildpacks conform to the [buildpack API](https://github.com/buildpacks/spec/blob/main/buildpack.md). +System buildpacks conform to the [buildpack API](https://github.com/buildpacks/spec/blob/main/buildpack.md). The `system.*pre*.buildpacks` will be provided to the lifecycle into a new file, `system.toml`. The `[system]` table in `system.toml` will be processed by the lifecycle, and each `pre`/`post` buildpack will run during the detect phase. Those that pass detection will run during the build phase. From 267a7c2392b68746ffc111d35e904e7e6c531070 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Fri, 4 Feb 2022 14:59:38 -0600 Subject: [PATCH 047/372] RFC 0099 [#198] Signed-off-by: Terence Lee --- text/{0000-slack.md => 0099-slack.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-slack.md => 0099-slack.md} (97%) diff --git a/text/0000-slack.md b/text/0099-slack.md similarity index 97% rename from text/0000-slack.md rename to text/0099-slack.md index ccad5b159..a0ece6a29 100644 --- a/text/0000-slack.md +++ b/text/0099-slack.md @@ -3,10 +3,10 @@ - Name: Migrate to CNCF Slack - Start Date: 2021-12-07 - Author(s): [@samj1912](https://github.com/samj1912) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#198](https://github.com/buildpacks/rfcs/pull/198) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 8873b7c3ace7eafc4f7ddedae78fee4f2a8b8912 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 4 Feb 2022 16:12:50 -0600 Subject: [PATCH 048/372] fix CNB_BP_ prefix Signed-off-by: Joe Kutner --- text/0000-buildpack-input-vars.md | 42 +++++++++++++++---------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md index ad258388d..a1bbe8d5a 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0000-buildpack-input-vars.md @@ -113,7 +113,7 @@ Executable: `/bin/build`, Working Dir: `` | Input | Attributes | Description |---------------------------|------------|---------------------------------- | `$0` | | Absolute path of `/bin/build` executable -| `$CNB_BP_LAYERS_DIR` | EIC | Absolute path to the buildpack layers directory +| `$CNB_LAYERS_DIR` | EIC | Absolute path to the buildpack layers directory | `$CNB_PLAN_PATH` | ER | Relevant [Buildpack Plan entries](#buildpack-plan-toml) from detection (TOML) | `$CNB_PLATFORM_DIR` | AR | Absolute path to the platform directory | `$CNB_PLATFORM_DIR/env/` | | User-provided environment variables for build @@ -124,23 +124,23 @@ Executable: `/bin/build`, Working Dir: `` | [exit status] | Success (0) or failure (1+) | Standard output | Logs (info) | Standard error | Logs (warnings, errors) -| `$CNB_BP_LAYERS_DIR/launch.toml` | App metadata (see [launch.toml](#launchtoml-toml)) -| `$CNB_BP_LAYERS_DIR/launch.sbom.` | Launch Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) -| `$CNB_BP_LAYERS_DIR/build.toml` | Build metadata (see [build.toml](#buildtoml-toml)) -| `$CNB_BP_LAYERS_DIR/build.sbom.` | Build Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) -| `$CNB_BP_LAYERS_DIR/store.toml` | Persistent metadata (see [store.toml](#storetoml-toml)) -| `$CNB_BP_LAYERS_DIR/.toml` | Layer metadata (see [Layer Content Metadata](#layer-content-metadata-toml)) -| `$CNB_BP_LAYERS_DIR/.sbom.` | Layer Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) -| `$CNB_BP_LAYERS_DIR//bin/` | Binaries for launch and/or subsequent buildpacks -| `$CNB_BP_LAYERS_DIR//lib/` | Shared libraries for launch and/or subsequent buildpacks -| `$CNB_BP_LAYERS_DIR//profile.d/` | Scripts sourced by Bash before launch -| `$CNB_BP_LAYERS_DIR//profile.d//` | Scripts sourced by Bash before launch for a particular process type -| `$CNB_BP_LAYERS_DIR//exec.d/` | Executables that provide env vars via the [Exec.d Interface](#execd) before launch -| `$CNB_BP_LAYERS_DIR//exec.d//` | Executables that provide env vars for a particular process type via the [Exec.d Interface](#execd) before launch -| `$CNB_BP_LAYERS_DIR//include/` | C/C++ headers for subsequent buildpacks -| `$CNB_BP_LAYERS_DIR//pkgconfig/` | Search path for pkg-config for subsequent buildpacks -| `$CNB_BP_LAYERS_DIR//env/` | Env vars for launch and/or subsequent buildpacks -| `$CNB_BP_LAYERS_DIR//env.launch/` | Env vars for launch (after `env`, before `profile.d`) -| `$CNB_BP_LAYERS_DIR//env.launch//` | Env vars for launch (after `env`, before `profile.d`) for the launched process -| `$CNB_BP_LAYERS_DIR//env.build/` | Env vars for subsequent buildpacks (after `env`) -| `$CNB_BP_LAYERS_DIR//*` | Other content for launch and/or subsequent buildpacks +| `$CNB_LAYERS_DIR/launch.toml` | App metadata (see [launch.toml](#launchtoml-toml)) +| `$CNB_LAYERS_DIR/launch.sbom.` | Launch Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) +| `$CNB_LAYERS_DIR/build.toml` | Build metadata (see [build.toml](#buildtoml-toml)) +| `$CNB_LAYERS_DIR/build.sbom.` | Build Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) +| `$CNB_LAYERS_DIR/store.toml` | Persistent metadata (see [store.toml](#storetoml-toml)) +| `$CNB_LAYERS_DIR/.toml` | Layer metadata (see [Layer Content Metadata](#layer-content-metadata-toml)) +| `$CNB_LAYERS_DIR/.sbom.` | Layer Software Bill of Materials (see [Software-Bill-of-Materials](#bill-of-materials)) +| `$CNB_LAYERS_DIR//bin/` | Binaries for launch and/or subsequent buildpacks +| `$CNB_LAYERS_DIR//lib/` | Shared libraries for launch and/or subsequent buildpacks +| `$CNB_LAYERS_DIR//profile.d/` | Scripts sourced by Bash before launch +| `$CNB_LAYERS_DIR//profile.d//` | Scripts sourced by Bash before launch for a particular process type +| `$CNB_LAYERS_DIR//exec.d/` | Executables that provide env vars via the [Exec.d Interface](#execd) before launch +| `$CNB_LAYERS_DIR//exec.d//` | Executables that provide env vars for a particular process type via the [Exec.d Interface](#execd) before launch +| `$CNB_LAYERS_DIR//include/` | C/C++ headers for subsequent buildpacks +| `$CNB_LAYERS_DIR//pkgconfig/` | Search path for pkg-config for subsequent buildpacks +| `$CNB_LAYERS_DIR//env/` | Env vars for launch and/or subsequent buildpacks +| `$CNB_LAYERS_DIR//env.launch/` | Env vars for launch (after `env`, before `profile.d`) +| `$CNB_LAYERS_DIR//env.launch//` | Env vars for launch (after `env`, before `profile.d`) for the launched process +| `$CNB_LAYERS_DIR//env.build/` | Env vars for subsequent buildpacks (after `env`) +| `$CNB_LAYERS_DIR//*` | Other content for launch and/or subsequent buildpacks From 05e2c15360013b244b8bbbee0c2176a88ca33929 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 10 Feb 2022 07:27:21 -0600 Subject: [PATCH 049/372] update input var names to address conflict with platform spec Signed-off-by: Joe Kutner --- text/0000-buildpack-input-vars.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md index a1bbe8d5a..5c0fc8183 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0000-buildpack-input-vars.md @@ -43,9 +43,9 @@ We will deprecate the positional arguments to `bin/detect` and `bin/build` with ### bin/build -* `CNB_LAYERS_DIR` - replaces the first positional argument to `bin/build`. **Note:** `CNB_LAYERS_DIR` conflicts with the platform spec, which will be updated to rename its `CNB_LAYERS_DIR` to `CNB_PARENT_LAYERS_DIR`. +* `CNB_LAYERS_DIR` - replaces the first positional argument to `bin/build`. **Note:** Uses the same env var name as the Platform spec, but refers to a different location. * `CNB_PLATFORM_DIR` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. -* `CNB_PLAN_PATH` - replaces the third positional argument to `bin/build`. Uses the same env var name as the Platform spec. +* `CNB_PLAN_PATH` - replaces the third positional argument to `bin/build`. **Note:** Uses the same env var name as the Platform spec, but refers to a different file. # How it Works [how-it-works]: #how-it-works @@ -58,10 +58,17 @@ cmd.Env = append(cmd.Env, EnvPlatformDir+"="+b.Dir) The positional arguments will be deprecated, but no warnings will be emitted if they are consumed. The lifecycle will continue to provide them to buildpack executable indefinitely, with no plan to remove them. +## Renaming Platform Env Vars + +Because some of the new buildpack input vars conflict with platform env var names, we intend to rename the platform env vars as part of a larger effort to refactor the platform spec. That will be convered in a future RFC. + +The conflict of env var names is strictly an experience problem. Because the env vars are used in different contexts, there is little risk that they will be reused by the internals of lifecycle or pack. + # Drawbacks [drawbacks]: #drawbacks - People have been using positional arguments to buildpacks for literally a decade +- New env var names conflict with platform env vars # Alternatives [alternatives]: #alternatives @@ -81,10 +88,6 @@ N/A # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes -## Platform Spec - -Rename `CNB_LAYERS_DIR` to `CNB_PARENT_LAYERS_DIR`. - ## Buildpack spec ### Detection From 6e15d37112895de121e97e22aabf7f933f75e50a Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 10 Feb 2022 07:32:56 -0600 Subject: [PATCH 050/372] update input var names to address conflict with platform spec Signed-off-by: Joe Kutner --- text/0000-buildpack-input-vars.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/0000-buildpack-input-vars.md b/text/0000-buildpack-input-vars.md index 5c0fc8183..66085f868 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0000-buildpack-input-vars.md @@ -45,7 +45,7 @@ We will deprecate the positional arguments to `bin/detect` and `bin/build` with * `CNB_LAYERS_DIR` - replaces the first positional argument to `bin/build`. **Note:** Uses the same env var name as the Platform spec, but refers to a different location. * `CNB_PLATFORM_DIR` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. -* `CNB_PLAN_PATH` - replaces the third positional argument to `bin/build`. **Note:** Uses the same env var name as the Platform spec, but refers to a different file. +* `CNB_BP_PLAN_PATH` - replaces the third positional argument to `bin/build`. # How it Works [how-it-works]: #how-it-works @@ -60,7 +60,7 @@ The positional arguments will be deprecated, but no warnings will be emitted if ## Renaming Platform Env Vars -Because some of the new buildpack input vars conflict with platform env var names, we intend to rename the platform env vars as part of a larger effort to refactor the platform spec. That will be convered in a future RFC. +Because `CNB_LAYERS_DIR` in the new input vars conflicts with a platform env var name, we intend to rename the platform env var as part of a larger effort to refactor the platform spec. That will be convered in a future RFC. The conflict of env var names is strictly an experience problem. Because the env vars are used in different contexts, there is little risk that they will be reused by the internals of lifecycle or pack. @@ -97,7 +97,7 @@ Executable: `/bin/detect`, Working Dir: `` | Input | Attributes | Description |---------------------------|------------|---------------------------------------------- | `$0` | | Absolute path of `/bin/detect` executable -| `$CNB_PLAN_PATH` | E | Absolute path to the build plan +| `$CNB_BUILD_PLAN_PATH` | E | Absolute path to the build plan | `$CNB_PLATFORM_DIR` | AR | Absolute path to the platform directory | `$CNB_PLATFORM_DIR/env/` | | User-provided environment variables for build | `$CNB_PLATFORM_DIR/#` | | Platform-specific extensions @@ -107,7 +107,7 @@ Executable: `/bin/detect`, Working Dir: `` | [exit status] | Pass (0), fail (100), or error (1-99, 101+) | Standard output | Logs (info) | Standard error | Logs (warnings, errors) -| `$CNB_PLAN_PATH` | Contributions to the the Build Plan (TOML) +| `$CNB_BUILD_PLAN_PATH` | Contributions to the the Build Plan (TOML) ### Build @@ -116,8 +116,8 @@ Executable: `/bin/build`, Working Dir: `` | Input | Attributes | Description |---------------------------|------------|---------------------------------- | `$0` | | Absolute path of `/bin/build` executable -| `$CNB_LAYERS_DIR` | EIC | Absolute path to the buildpack layers directory -| `$CNB_PLAN_PATH` | ER | Relevant [Buildpack Plan entries](#buildpack-plan-toml) from detection (TOML) +| `$CNB_LAYERS_DIR` | EIC | Absolute path to the buildpack layers directory +| `$CNB_BP_PLAN_PATH` | ER | Relevant [Buildpack Plan entries](#buildpack-plan-toml) from detection (TOML) | `$CNB_PLATFORM_DIR` | AR | Absolute path to the platform directory | `$CNB_PLATFORM_DIR/env/` | | User-provided environment variables for build | `$CNB_PLATFORM_DIR/#` | | Platform-specific extensions From 884eaef5582e5c30b8cacbcdbdd82514f70edc24 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 16 Feb 2022 14:30:35 -0500 Subject: [PATCH 051/372] Clarify Dockerfile restrictions Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 22b1eeabc..2c76f9130 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -102,7 +102,7 @@ Support for other instruction formats, e.g., LLB JSON files, could be added in t If no Dockerfiles are present, `/bin/build` may still consume build plan entries and add metadata to `build.toml`/`launch.toml`. Dockerfiles are applied to their corresponding base images after all extensions are executed and before any regular buildpacks are executed. -Dockerfiles are applied in the order determined during buildpack detection. +Dockerfiles are applied in the order determined during buildpack detection. When multiple Dockerfiles are applied, the intermediate image generated from the application of the current Dockerfile will be provided as the `base_image` ARG to the next Dockerfile. Dockerfiles that target the run image (only) may ignore the provided `base_image` (e.g., `FROM some-other-image`). Dockerfiles that change the runtime base image may still use `COPY --from=${base_image}`. All Dockerfiles are provided with `base_image` and `build_id` args. The `base_image` arg allows the Dockerfile to reference the original base image. From 9ec8c04303a201cdaa617498f6d26ba25d142072 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Wed, 23 Feb 2022 16:49:06 -0600 Subject: [PATCH 052/372] RFC 0100 [#190] Signed-off-by: Jesse Brown --- ...buildpack-input-vars.md => 0100-buildpack-input-vars.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-buildpack-input-vars.md => 0100-buildpack-input-vars.md} (96%) diff --git a/text/0000-buildpack-input-vars.md b/text/0100-buildpack-input-vars.md similarity index 96% rename from text/0000-buildpack-input-vars.md rename to text/0100-buildpack-input-vars.md index 66085f868..fa5281615 100644 --- a/text/0000-buildpack-input-vars.md +++ b/text/0100-buildpack-input-vars.md @@ -3,10 +3,10 @@ - Name: Replace positional args to Buildpack executables with env vars - Start Date: 2021-11-18 - Author(s): [jkutner](https://github.com/jkutner) -- Status: Draft Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#190](https://github.com/buildpacks/rfcs/pull/190) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/lifecycle#806](https://github.com/buildpacks/lifecycle/issues/806), [buildpacks/lifecycle#807](https://github.com/buildpacks/lifecycle/issues/807) - Supersedes: N/A # Summary From 1a9018dcbadd888e25b716501a0ea85f00ada72d Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 24 Feb 2022 17:15:57 -0500 Subject: [PATCH 053/372] Use SOURCE_DATE_EPOCH to configure image create time Signed-off-by: Natalie Arellano --- text/0000-source-date-epoch.md | 84 ++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 text/0000-source-date-epoch.md diff --git a/text/0000-source-date-epoch.md b/text/0000-source-date-epoch.md new file mode 100644 index 000000000..fa62843b4 --- /dev/null +++ b/text/0000-source-date-epoch.md @@ -0,0 +1,84 @@ +# Meta +[meta]: #meta +- Name: Use SOURCE_DATE_EPOCH to configure image create time +- Start Date: 2022-02-24 +- Author(s): natalieparellano +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +To allow for [build reproducibility](https://github.com/buildpacks/spec/blob/main/platform.md#build-reproducibility), images created by Cloud Native Buildpacks have a hard-coded create time of January 1, 1980. We have received requests from the community (see [here](https://github.com/buildpacks/pack/issues/1281) and [here](https://buildpacks.slack.com/archives/C94UJCNV6/p1643842965830459)) to allow for this value to be configurable. This RFC proposes making the value configurable by setting `SOURCE_DATE_EPOCH` in the lifecycle's execution environment during `export`. + +# Definitions +[definitions]: #definitions + +* build reproducibility: identical inputs should produce identical outputs ([blog post](https://medium.com/buildpacks/time-travel-with-pack-e0efd8bf05db)) +* `SOURCE_DATE_EPOCH`: a "standardised environment variable that distributions can set centrally and have build tools consume this in order to produce reproducible output" (see [here](https://reproducible-builds.org/docs/source-date-epoch/)) + +# Motivation +[motivation]: #motivation + +- Why should we do this? + Users have asked for it +- What use cases does it support? + A meaningful image create time, e.g., the ability to determine which images are most recent +- What is the expected outcome? + Images built with `SOURCE_DATE_EPOCH` set will not be reproducible, as the config blob will change + +# What it is +[what-it-is]: #what-it-is + +- Define the target persona: buildpack user, platform operator + * A buildpack user could pass a flag to `pack` to set `SOURCE_DATE_EPOCH` to the current time in the lifecycle's execution environment during `export`. + * A platform operator could choose to set `SOURCE_DATE_EPOCH` whenever `export` is run. + +- If applicable, describe the differences between teaching this to existing users and new users. + We should mention this feature in the lifecycle release notes and docs. + +# How it Works +[how-it-works]: #how-it-works + +This [PR](https://github.com/buildpacks/imgutil/pull/137) to imgutil would read the environment variable at the point of saving the image. An alternative implementation could be to have the `exporter` read the variable and provide it via a new `SetCreatedAt()` method on the `imgutil.Image` interface. The latter might be safer as it would avoid unintended side effects for other consumers of imgutil (e.g., `pack`). + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +It breaks build reproducibility. However platforms must explicitly opt-in to this feature. + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + * Doing nothing + * Allowing layer timestamps to also be configurable, but this would effectively prohibit launch layer re-use and make builds slower +- Why is this proposal the best? + * It is easy to implement, easy to use, and solves for the desired use case +- What is the impact of not doing this? + * The lack of meaningful image create times makes it difficult to automate cleanup of images and could be a deal breaker for some users + +# Prior Art +[prior-art]: #prior-art + +* See under "Reading the variable" [here](https://reproducible-builds.org/docs/source-date-epoch/) +* [ko](https://github.com/google/ko#why-are-my-images-all-created-in-1970) +* [jib](https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#why-is-my-image-created-48-years-ago) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? + * Should we require users to be on platform api 0.9 in order to use this feature? (Opinion: no, because this feature is invisible if the environment variable is not set. The motivation for spec'ing this in the platform api is to make it clear for platform operators how to use it.) +- What parts of the design do you expect to be resolved through implementation of the feature? + * Should we do this in imgutil or the lifecycle? (see above) + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +See [spec PR](https://github.com/buildpacks/spec/pull/292). From 04ed9535a6fdd24db27a2837d14052a000e77d22 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Thu, 2 Dec 2021 15:36:51 +0000 Subject: [PATCH 054/372] Add image history metadata Signed-off-by: Sambhav Kothari --- text/0000-history.md | 85 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 text/0000-history.md diff --git a/text/0000-history.md b/text/0000-history.md new file mode 100644 index 000000000..ad456f8bb --- /dev/null +++ b/text/0000-history.md @@ -0,0 +1,85 @@ +# Meta +[meta]: #meta +- Name: Buildpack layer metadata +- Start Date: 2021-12-02 +- Author(s): [@samj1912](https://github.com/samj1912) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +Add layer history metadata to the output image for buildpack and app layers to improve visualization in tools like `DockerHub`, `dive` etc. + +# Definitions +[definitions]: #definitions + +`history`: The history key in the `OCI` config. See [properties](https://github.com/opencontainers/image-spec/blob/main/config.md#properties) + +# Motivation +[motivation]: #motivation + +This will allow Buildpack built images to be better visualized by common container visualization and debugging tools. + +# What it is +[what-it-is]: #what-it-is + +Various container image introspection tools look at `history.created_by` for each layer to visualize the layer. We should populate this key with a value that describes where a layer came from, for buildpacks this can be - + +- `Layer: {{ buildpack.layer.name }}, Created by: {{ buildpack.id }}@{{ buildpack.version }}` + +for app layers, this can be - + +- `Application Slice: {{ slice_number }} Created by: {{ buildpack.id }}@{{ buildpack.version }}` + +Where `slice_number` is just an integer number starting with `1` and the `buildpack.id` is the id of the buildpack that specified the application slice. + +for config layer, this can be - + +- `Buildpacks Launcher Config` + +for SBOM layer, this can be - + +- `Software Bill-of-Materials` + +for launcher this can be - + +- `Buildpacks Application Launcher` + +for the process types layer this can be - + +- `Buildpacks Process Types` + +For the base image, the lifecycle should copy the existing history to the output image. +# How it Works +[how-it-works]: #how-it-works + +The lifecycle adds the above metadata to the output config blob. + +# Drawbacks +[drawbacks]: #drawbacks + +More complexity? + +# Alternatives +[alternatives]: #alternatives + +Allow buildpacks to create image history metadata. + +# Prior Art +[prior-art]: #prior-art + +N/A + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +Noted above. From cd9471f7c8f0c84a14d5b2d9a1111979808e4df7 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Tue, 1 Mar 2022 16:25:05 -0500 Subject: [PATCH 055/372] Move extensions out of order table Signed-off-by: Stephen Levine Co-authored-by: Natalie Arellano --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 2c76f9130..f66f7c30f 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -74,7 +74,7 @@ Extensions participate in the buildpack detection process, with the same UID, GI However, - `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. - Extensions may only output `provides` entries to the build plan. They must not output `requires`. -- Extensions must all precede regular buildpacks in `order` definitions (e.g., in `builder.toml`). +- Extensions are not included in `order` definitions (e.g., in `builder.toml`); instead, a separate `order-ext` table should be used. The `order-ext` table will be prepended to the provided `order` (as if `order-ext` were a meta-buildpack). - Extensions are always `optional`. Extensions generate Dockerfiles before the regular buildpack build phase. From c6c99c772086be16b6e82f4bc5f804d3d07d11e9 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Fri, 4 Mar 2022 14:18:25 -0500 Subject: [PATCH 056/372] RFC 0101 [#179] Signed-off-by: Emily Casey --- ...cks-in-builder.md => 0101-system-buildpacks-in-builder.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename text/{0000-system-buildpacks-in-builder.md => 0101-system-buildpacks-in-builder.md} (97%) diff --git a/text/0000-system-buildpacks-in-builder.md b/text/0101-system-buildpacks-in-builder.md similarity index 97% rename from text/0000-system-buildpacks-in-builder.md rename to text/0101-system-buildpacks-in-builder.md index 645a14299..18a912404 100644 --- a/text/0000-system-buildpacks-in-builder.md +++ b/text/0101-system-buildpacks-in-builder.md @@ -3,9 +3,9 @@ - Name: System Buildpacks in Builder Images - Start Date: 2021-07-24 - Author(s): [@jkutner](https://github.com/jkutner) -- RFC Pull Request: (leave blank) +- RFC Pull Request: [rfcs#179](https://github.com/buildpacks/rfcs/pull/179) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/rfcs#209](https://github.com/buildpacks/rfcs/issues/209) - Supersedes: N/A # Summary From d2b9307c2db490ff608241f409209b892b733ad6 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Mar 2022 21:00:47 -0600 Subject: [PATCH 057/372] Fix typo Referred to `bin/build` when it should refer to `bin/detect`. Signed-off-by: Mikey Boldt --- text/0100-buildpack-input-vars.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0100-buildpack-input-vars.md b/text/0100-buildpack-input-vars.md index fa5281615..27782d1dc 100644 --- a/text/0100-buildpack-input-vars.md +++ b/text/0100-buildpack-input-vars.md @@ -38,8 +38,8 @@ We will deprecate the positional arguments to `bin/detect` and `bin/build` with ### bin/detect -* `CNB_PLATFORM_DIR` - replaces the first positional argument to `bin/build`. Uses the same env var name as the Platform spec. -* `CNB_BUILD_PLAN_PATH` - replaces the second positional argument to `bin/build`. Uses the same env var name as the Platform spec. +* `CNB_PLATFORM_DIR` - replaces the first positional argument to `bin/detect`. Uses the same env var name as the Platform spec. +* `CNB_BUILD_PLAN_PATH` - replaces the second positional argument to `bin/detect`. Uses the same env var name as the Platform spec. ### bin/build From 09517a01b787aab6c6127b23c5ba4ad2059d70de Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 4 Mar 2022 22:47:01 +0000 Subject: [PATCH 058/372] Bump actions/checkout from 2 to 3 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/issues-generation.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/issues-generation.yml b/.github/workflows/issues-generation.yml index ba2a4828a..a66230ea9 100644 --- a/.github/workflows/issues-generation.yml +++ b/.github/workflows/issues-generation.yml @@ -15,7 +15,7 @@ jobs: if: ${{ github.event.issue.pull_request || github.event.pull_request }} runs-on: ubuntu-latest steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 - name: Run Issue Generation uses: jromero/issue-generation-action@v1.0.0-beta.4 id: issues-generation From 32ac0ed2aaa3b3c0b0de09caac83309349ff6979 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 10 Mar 2022 09:37:48 -0500 Subject: [PATCH 059/372] Update text/0000-source-date-epoch.md Signed-off-by: Natalie Arellano Co-authored-by: Emily Casey --- text/0000-source-date-epoch.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-source-date-epoch.md b/text/0000-source-date-epoch.md index fa62843b4..341855cbb 100644 --- a/text/0000-source-date-epoch.md +++ b/text/0000-source-date-epoch.md @@ -50,7 +50,7 @@ This [PR](https://github.com/buildpacks/imgutil/pull/137) to imgutil would read Why should we *not* do this? -It breaks build reproducibility. However platforms must explicitly opt-in to this feature. +Platforms that choose to set SOURCE_DATE_EPOCH to real creation time will not have 100% reproducible builds. # Alternatives [alternatives]: #alternatives From aa536d0353f94875798c0dd8b1624a80d40f0116 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Mon, 14 Mar 2022 07:56:14 +0000 Subject: [PATCH 060/372] Propose alternative project scaffolding in pack Allow pack to promote libcnb project scaffolding and support 3rd parties to provide their own buildpack scaffolding. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 210 +++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 text/0000-pack-buildpack-new-alternatives.md diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md new file mode 100644 index 000000000..2a8be7891 --- /dev/null +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -0,0 +1,210 @@ +# Meta +[meta]: #meta +- Name: Pack Buildpack New to Support Alternatives +- Start Date: 2022-03-10 +- Author(s): aidandelaney +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal extends `pack buildpack new` with a `--libcnb` option and a `--template URL` option to allow alternatives to the shell-script template. The `--libcnb` option creates a new buildpack written in golang using `libcnb`. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. + +# Definitions +[definitions]: #definitions + +* **project scaffolding**: minimal code and file system structure used to start a project. + +# Motivation +[motivation]: #motivation + +The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal `libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. + +Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--target URL` allows other projects to re-use `pack` as their scaffolding tool. + +`pack buildpack new --libcnb` supports end-users who wish to scaffold a new `libcnb`-based buildpack. The operation of `pack buildpack new --libcnb` does not require internet access to succeed. `pack buildpack new --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. + +Extending `pack buildpack new` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. + +# What it is +[what-it-is]: #what-it-is + +Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](https://yeoman.io/) [in](https://github.com/kowainik/summoner) [other](https://crates.io/crates/cargo-scaffold) [ecosystems](https://github.com/golang-standards/project-layout). Scaffolding systems are employed to ease onboarding of new developers. Within `pack` this feature is targeted at onboarding buildpack authors. + +- Explaining the feature largely in terms of examples. + +`pack buildpack new --libcnb` should generate skeleton code according to the [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: + +```bash +. +├── buildpack.toml +├── cmd +│ └── main.go +├── go.mod +├── go.sum +└── pkg + ├── build.go + └── detect.go +``` + +This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the `--libcnb` skeleton. `pack buildpack new --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with `--libcnb` onboarding and support experienced buildpack authors to use the scaffolding of their choice. + +# How it Works +[how-it-works]: #how-it-works + +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from an embedded file system. + +A full invocation to generate `libcnb` scaffolding is similar to the [documented project scaffolding](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). + +```bash +pack buildpack new --libcnb examples/ruby \ + --api 0.7 \ + --path ruby-buildpack \ + --version 0.0.1 \ + --stacks io.buildpacks.samples.stacks.bionic +``` + +A full session includes the terminal prompts that the project scaffolding tool asks of the end user: + +```bash +pack buildpack new --libcnb examples/ruby \ + --api 0.7 \ + --path ruby-buildpack \ + --version 0.0.1 \ + --stacks io.buildpacks.samples.stacks.bionic +Package name (often github.com/username/repo)? github.com/AidanDelaney/ruby +Created project in ruby-buildpack +``` + +Templates are provided as a file tree. A root-level `prompts.yaml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and golang skeleton code is: + +```bash +. +├── prompts.yaml +├── cmd + └── main.go +``` + +Where `prompts.yaml` is of the form: + +```yaml +prompts: + - variable: packageName + prompt: Package name (often github.com/username/repo) +``` + +And an example templated source file is: + +```golang +package main + +import ( + "github.com/buildpacks/libcnb" + + bp "{{ .packageName }}/pkg" +) + +func main() { + libcnb.Main( + bp.Detect{}, + bp.Build{}, + ) +} +``` + +The following template variables, known as the _reserved template variables_ are available for substitution in all project files: + +* `.ID` -- the buildpack id that will be written to `buildpack.toml` obtained from a positional argument to `pack` +* `.Version` -- the buildpack version obtained from the argument to the `--version` flag +* `.API` -- the buildpack API version obtained from the argument to the `--api` flag +* `.Stacks` -- a slice containing the buildpack stacks obtained from the argument to the `--stacks` flag + +Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file and not redefine reserved template variables. An optional default value may be provided. Variables that do not provide a default value must + +```yaml +prompts: + - variable: packageName + prompt: Package name (often github.com/username/repo) + default: DefaultValue +``` + +Answers to prompts can be provided in a yaml file. This supports programmatic use of `pack`: + +``` +pack buildpack new --libcnb examples/ruby \ + --config-file answers.yaml \ + --api 0.7 \ + --path ruby-buildpack \ + --version 0.0.1 \ + --stacks io.buildpacks.samples.stacks.bionic +Created project in ruby-buildpack +``` + +The format of `--config-file` parallels the format used in `cookiecutter`. An example of which is: + +```yaml +defaultContext: + packageName: com.example/an-example +``` + +All input files are expected to be normalized to Unix line endings. + +# Migration +[migration]: #migration + +The current `bash` project scaffolding can be embedded in `pack` as an embedded file system. This would allow the `--libcnb` and `--target URL` code to be reused. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +* Golang project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. +* Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpaks/bash my_buildpack`. +* Including `--libcnb` project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of the embedded file system plus the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). The `pack` binary will grow to +* This proposal commits to support a specific project scaffolding format. A migration path should be established if and when a de-facto standard golang template library becomes available. +* In supporting `--libcnb` we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + +We have considered a wider re-implementation of [cookiecutter](https://cookiecutter.readthedocs.io/). However, the scope of a re-implementation was considered too wide. In addition, we have considered a cookiecutter-lite implementation where project scaffolding is cloned from a repository. This proposal recommends a project scaffolding that can be embedded in `pack` and, in addition, can be downloaded from an Internet source. + +We have also considered [springerle](https://github.com/carlmjohnson/springerle) which uses a single txtar file to describe skeleton project structure. The use of a single txtar file requires template providers to write and maintain this format. + +[bare](https://github.com/bare-cli/bare) is a new project. It assumes that all templates are stored on github.com. + +- Why is this proposal the best? +To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that embeds cnb-provided skeletons in `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. + +- What is the impact of not doing this? + +Omitting support for `libcnb` project scaffolding requires new buildpack authors to consult our documentation about project structure. Moreover, as `pack` supports scaffolding of shell-script buildpacks the impression is given that the buildpacks project _prefers_ shell implementations. + +# Prior Art +[prior-art]: #prior-art + +There are many, many competing implementations of project scaffolding tools. + +* Python's [Cookiecutter](https://cookiecutter.readthedocs.io/): widely used to scaffold projects in many languages, including golang. Requires a Python runtime to be available. +* [springerle](https://github.com/carlmjohnson/springerle): golang re-think of cookiecutter. Springerle uses a single txtar file augmented with a header containing user prompts. This proposal prefers using a filesystem rather than a single txtar file as the filesystem approach extends to cloning repositories. +* [cgapp](https://github.com/create-go-app/cli): This proposal heavily borrows from cgapp. It is necessary to modify parts of cgapp in order to apply variable substitution. +* JavaScript's [Yeoman](https://yeoman.io/): widely used in the web ecosystem +* [boilr](https://github.com/tmrts/boilr/): golang cookiecutter. Moribund project. +* [bare](https://github.com/bare-cli/bare): golang cookiecutter. Possible successor to boilr. Both boilr and bare assume the templates are stored on github.com and use the zip downloading functionality of that specific service. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +This proposal does not require any spec changes. From 4308429eb04a0173eb9202210d9a230b55d00834 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Tue, 15 Mar 2022 18:27:56 +0530 Subject: [PATCH 061/372] Add Sam to BAT config Signed-off-by: Sambhav Kothari --- .github/auto-assign-bat.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/auto-assign-bat.yml b/.github/auto-assign-bat.yml index 7e6812205..f61b0b51a 100644 --- a/.github/auto-assign-bat.yml +++ b/.github/auto-assign-bat.yml @@ -8,3 +8,4 @@ filterLabels: assignees: - ekcasey - hone +- samj1912 From 617df9d15af287e2387e7d1e72352d842c4cfa8f Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 23 Mar 2022 09:51:29 -0400 Subject: [PATCH 062/372] RFC 0102 [#194] Signed-off-by: Natalie Arellano --- text/{0000-history.md => 0102-history.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-history.md => 0102-history.md} (93%) diff --git a/text/0000-history.md b/text/0102-history.md similarity index 93% rename from text/0000-history.md rename to text/0102-history.md index ad456f8bb..cbcb3dfb3 100644 --- a/text/0000-history.md +++ b/text/0102-history.md @@ -3,10 +3,10 @@ - Name: Buildpack layer metadata - Start Date: 2021-12-02 - Author(s): [@samj1912](https://github.com/samj1912) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#194](https://github.com/buildpacks/rfcs/pull/194) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 146bafd3704124afb767a2966d159eb2cad02961 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 23 Mar 2022 10:50:23 -0500 Subject: [PATCH 063/372] RFC 0103 [#204] Signed-off-by: Terence Lee --- ...{0000-source-date-epoch.md => 0103-source-date-epoch.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-source-date-epoch.md => 0103-source-date-epoch.md} (95%) diff --git a/text/0000-source-date-epoch.md b/text/0103-source-date-epoch.md similarity index 95% rename from text/0000-source-date-epoch.md rename to text/0103-source-date-epoch.md index 341855cbb..284fe997f 100644 --- a/text/0000-source-date-epoch.md +++ b/text/0103-source-date-epoch.md @@ -3,10 +3,10 @@ - Name: Use SOURCE_DATE_EPOCH to configure image create time - Start Date: 2022-02-24 - Author(s): natalieparellano -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#204](https://github.com/buildpacks/rfcs/pull/204) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/lifecycle#809](https://github.com/buildpacks/lifecycle/issues/809) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From ed869df9ff700c5687a35c2963fd41a5a124206d Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 23 Mar 2022 14:35:17 -0400 Subject: [PATCH 064/372] Add template for tracking issues Signed-off-by: Natalie Arellano --- ISSUE_TEMPLATE/tracking.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 ISSUE_TEMPLATE/tracking.md diff --git a/ISSUE_TEMPLATE/tracking.md b/ISSUE_TEMPLATE/tracking.md new file mode 100644 index 000000000..fdcdd42a6 --- /dev/null +++ b/ISSUE_TEMPLATE/tracking.md @@ -0,0 +1,25 @@ +--- +name: Tracking Issue +about: Open an issue to track progress toward implementation of a merged RFC +title: '[RFC #] - ' +labels: type/tracking +assignees: '' + +--- + +[RFC #](https://github.com/buildpacks/rfcs/blob/main/text/) - + +Spec: +- [ ] https://github.com/buildpacks/spec/issues/ +- [ ] Released in API version `` + +Lifecycle: +- [ ] https://github.com/buildpacks/lifecycle/issues/ +- [ ] Released in lifecycle version `` + +Pack: +- [ ] https://github.com/buildpacks/pack/issues/ +- [ ] Released in pack version `` + +Documentation: +- [ ] https://github.com/buildpacks/docs/issues/ From fa77dacad0dbb64762df76200bd6f9a6edb52eeb Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 23 Mar 2022 16:57:46 -0400 Subject: [PATCH 065/372] Add placeholder for other repositories besides the ones noted Signed-off-by: Natalie Arellano --- ISSUE_TEMPLATE/tracking.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ISSUE_TEMPLATE/tracking.md b/ISSUE_TEMPLATE/tracking.md index fdcdd42a6..acec31bb5 100644 --- a/ISSUE_TEMPLATE/tracking.md +++ b/ISSUE_TEMPLATE/tracking.md @@ -23,3 +23,7 @@ Pack: Documentation: - [ ] https://github.com/buildpacks/docs/issues/ + +: +- [ ] +- [ ] Released in version `` From be9dc4dd5a5630437d5696cce788789cffd09671 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 24 Mar 2022 09:27:18 -0400 Subject: [PATCH 066/372] Add libcnb Signed-off-by: Natalie Arellano --- ISSUE_TEMPLATE/tracking.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ISSUE_TEMPLATE/tracking.md b/ISSUE_TEMPLATE/tracking.md index acec31bb5..e9e1d470e 100644 --- a/ISSUE_TEMPLATE/tracking.md +++ b/ISSUE_TEMPLATE/tracking.md @@ -17,6 +17,10 @@ Lifecycle: - [ ] https://github.com/buildpacks/lifecycle/issues/ - [ ] Released in lifecycle version `` +Libcnb: +- [ ] https://github.com/buildpacks/libcnb/issues/ +- [ ] Released in libcnb version `` + Pack: - [ ] https://github.com/buildpacks/pack/issues/ - [ ] Released in pack version `` From 7288dd562ed622d7521f01f262d103a767b57919 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 31 Mar 2022 08:53:27 -0500 Subject: [PATCH 067/372] Stop Deleting Cache Images Signed-off-by: Jesse Brown --- text/0000-stop-deleting-cache-images.md | 73 +++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 text/0000-stop-deleting-cache-images.md diff --git a/text/0000-stop-deleting-cache-images.md b/text/0000-stop-deleting-cache-images.md new file mode 100644 index 000000000..edd35bd70 --- /dev/null +++ b/text/0000-stop-deleting-cache-images.md @@ -0,0 +1,73 @@ +# Meta +[meta]: #meta +- Name: Stop deleting cache images +- Start Date: 2022-03-31 +- Author(s): jabrown85 +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: [buildpacks/lifecycle#803](https://github.com/buildpacks/lifecycle/issues/803) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +Lifecycle will no longer try to delete previous cache images when publishing cache images. + +# Definitions +[definitions]: #definitions + +* Cache Image: The cache stored between builds - stored in a registry. +* ECR: Amazon's container registry product + +# Motivation +[motivation]: #motivation + +- Why should we do this? + As discussed in [buildpacks/lifecycle#803](https://github.com/buildpacks/lifecycle/issues/803), some registries (ECR) do not support `DELETE`. For platforms that work exclusively with such registries, the warning output by lifecycle and the time taken to fail is unavoidable and lifecycle is wasting time trying to complete an operation that will never succeed. + +- What use cases does it support? + All platforms that use cache images against registries that do not support delete will no longer see warning messages. + +- What is the expected outcome? + Platforms will need to handle cleanup of their cache images on their own, if they desire. + +# What it is +[what-it-is]: #what-it-is + +Lifecycle will no longer attempt to delete cache images during cache image export. + +# How it Works +[how-it-works]: #how-it-works + +Lifecycle will no longer attempt to delete cache images during cache image export. + +# Migration +[migration]: #migration + +Lifecycle will document this behavior change in Release Notes/Changelog along with the associated Platform API that enables the new behavior. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +Platform authors relying on this behavior will need to take additional measures to ensure cache image cleanup. + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + * Add regex or configuration to drive registry hosts to ignore during DELETE * +- Why is this proposal the best? + * Lifecycle is not currently cleaning up any other resources + * Deleting the cache images can hurt reproducibility + * There are more public registries that don't allow DELETE +- What is the impact of not doing this? + * End users continue seeing warnings the platform can do nothing about. + + +# Prior Art +[prior-art]: #prior-art + +Discussion at [buildpacks/lifecycle#803](https://github.com/buildpacks/lifecycle/issues/803). From e58a18334470fcb9780e91f3e068d73a34de5775 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 31 Mar 2022 08:53:27 -0500 Subject: [PATCH 068/372] fixup! Stop Deleting Cache Images Signed-off-by: Jesse Brown --- text/0000-stop-deleting-cache-images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-stop-deleting-cache-images.md b/text/0000-stop-deleting-cache-images.md index edd35bd70..0fe8f5653 100644 --- a/text/0000-stop-deleting-cache-images.md +++ b/text/0000-stop-deleting-cache-images.md @@ -52,7 +52,7 @@ Lifecycle will document this behavior change in Release Notes/Changelog along wi Why should we *not* do this? -Platform authors relying on this behavior will need to take additional measures to ensure cache image cleanup. +Platform authors relying on this behavior will need to take additional measures to ensure cache image cleanup or the destination registry will continue to grow. # Alternatives [alternatives]: #alternatives From 4badd26916dd9a1dfa78f0d3cdd3d3fc6a51ea21 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 31 Mar 2022 08:53:27 -0500 Subject: [PATCH 069/372] added alternative Signed-off-by: Jesse Brown --- text/0000-stop-deleting-cache-images.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-stop-deleting-cache-images.md b/text/0000-stop-deleting-cache-images.md index 0fe8f5653..b90027bfe 100644 --- a/text/0000-stop-deleting-cache-images.md +++ b/text/0000-stop-deleting-cache-images.md @@ -59,6 +59,7 @@ Platform authors relying on this behavior will need to take additional measures - What other designs have been considered? * Add regex or configuration to drive registry hosts to ignore during DELETE * + * Stop deleting cache images by default in newer platform API versions, but add a platform-level configuration to enable previous behavior. - Why is this proposal the best? * Lifecycle is not currently cleaning up any other resources * Deleting the cache images can hurt reproducibility From de9184a1cdd43823847e8c4218dc893e44dc9778 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 11 May 2022 08:16:00 -0400 Subject: [PATCH 070/372] New RFC Process Signed-off-by: Emily Casey --- text/0000-rfc-process.md | 162 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 text/0000-rfc-process.md diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md new file mode 100644 index 000000000..fad9f4a8b --- /dev/null +++ b/text/0000-rfc-process.md @@ -0,0 +1,162 @@ +# Meta +[meta]: #meta +- Name: RFC Process +- Start Date: 2021-05-10 +- Author(s): [@ekcasey](https://github.com/ekcasey) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: [RFC 0004](0004-rfc-process.md) + +# Summary +[summary]: #summary + +The "RFC" (request for comments) process provide a consistent and controlled path for new features and other substantive changes to enter the project. + +The RFC process: +* provides a set of checkpoints that ensure changes align with the overall technical and strategic vision for the process. +* ensures the motivation for a change is clear. +* ensures the impact of a change on users is clear and migration path and backwards compatibility are considered. +* aligns stakeholders on any changes to the user interface(s) and/or APIs (e.g. pack user interface, platform API, buildpack API). +* aligns stakeholders on any substantive architectural decision changes +* Aligns stakeholders on any processes or workflows adopted by the project. +* provides visibility to the community regarding on-going work and upcoming changes. +* provides a mechanism by which any interested party can provide early feedback on an upcoming change. +* is open to anyone! We enthusiastically welcome RFCs from authors that have no formal role in project governance (yet ;p). + +The RFC process **is not**: +* A replacement for high-quality user-facing documentation. +* A transaction log of changes. Readers should be able to understand the change proposed without undue reference to previous RFCs. +* A feature request process. RFCs require a level of design and implementation detail that goes beyond a feature request. Pure feature requests should instead be initiated as discussions on the community repo, issues on this repo, or issues on component repos. These requests may serve as the motivation for future RFCs. + + +# Definitions +[definitions]: #definitions + +**Technical Oversight Committee**: The governing body responsible for overseeing the project as described in our governance document. + +**Team**: A group responsible for a particular area of the project. + +**Team Lead**: A team maintainer with special responsibilities for representing the concerns of a team to the project, including casting binding votes on project RFCs. + +**RFC**: A document proposing a substantive change to the project as described in this document. + +**Project RFC**: An RFC with cross-cutting implications, requiring collaboration between multiple teams or affecting multiple personas. + +**Team RFC**: An RFC with narrower implication in comparison to a project RFC, with work scoped to a single team and implications for a narrower set of personas. + +**Call for Votes**: When an RFC is deemed ready by a team lead, that team lead initiates the voting process with a call for votes. At this point the RFC is closed to modification. + +**End Date**: When a call for votes is initiated, an end date for voting is set. Any person wishing to vote on an RFC must do so by the end date. + +**Lazy Consensus**: Voting on project level RFCs is done by lazy consensus. Any project member with a binding vote who has not voted by the end date is assumed to assent to the RFC. + +**Binding Vote**: Binding Votes have formal power within the RFC process. A single no vote by a project member with a binding vote is sufficient to reject an RFC. + +**Non-Binding Vote**: Anyone from the project or community may cast a non-binding vote on an RFC to express their position. While these votes do not have formal power they are taken seriously and may affect the votes of members with binding votes. + + +# Motivation +[motivation]: #motivation + +This RFC process is an evolution of our [previous process](0004-rfc-process.md). In comparison to the previous process this proposal makes two substantive cha + +# What it is +[what-it-is]: #what-it-is + +### What Types of Changes Require an RFC? + +#### Project vs Team RFCs + +### Process + +#### Drafting +All RFCs begin life as a draft. Anyone wishing to propose a change to the project should create a draft RFC by: + +- Fork the RFC repo: +- Copy `0000-template.md` to `text/0000-my-feature.md` (where 'my-feature' is descriptive. don't assign an RFC number yet). +- Fill in RFC. Any section can be marked as "N/A" if not applicable. +- Submit a pull request. + +#### Finding a Steward +All RFCs, even project RFCs "belong" to a team. The team lead of the responsible team is the steward of the RFC. The author and the steward of an RFC may, at times, be the same person. + +In many but not all cases the appropriate team to steward an RFC will be obvious. When the appropriate steward is not obvious the author should work with the community to find a home within one of the teams. Factors to consider when finding home include: +* Which team has the most relevant technical context? +* Which team has the deepest understanding of the use-case and the needs of impacted personas? +* Which team is responsible for the components most impacted by the proposed change? +* Which team is enthusiastic about supporting the change? + +If a home cannot be found for a draft RFC it remains in draft until one can be found. This does not necessary mean that the RFC is not a good idea or not something the project will take up eventually. It may simply be that the project does not have the bandwidth to prioritize this particular time. + +#### Stewardship + +The steward and their team should: + * work with the author of the RFC to ensure that the RFC is complete and implementable contingent upon approval. This can happen synchronously at team working groups or asynchronously via github and slack. + * raise visibility to and solicit feedback from other stakeholders including the TOC, other teams, and the community at large. This can happen synchronously at the project working group and asynchronously vai github and slack. + * drive consensus for the RFC by incorporating feedback from stakeholders so that the RFC has the best possible chance of approval during the voting process. + * ensure there is a plan in place to implement the RFC in a reasonable time frame, contingent upon approval. The team itself need not implement the RFC but we should not approve RFC for which we have no concrete plan to implement. + * in the case of complicated or risky RFCs, a POC should be developed at this stage to validate and de-risk the proposed design. + +#### Voting + +Ideally the voting should be a formality and not a moment to discover new disagreement, consensus already haven been driven by the steward. + +When the steward deems the RFC ready and likely to be accepted they should formally call for votes and set an end date for voting. This process does not prescribe a length for the voting window, but stewards should make a good faith effort to ensure that all interested parties, and in particular those with binding votes, have adequate opportunity to read and cast votes. + +For project RFCs, all TOC members and all team leads are given a binding vote. + +For team RFCs, all team maintainers are given a binding vote. + +Votes are cast via reviews on the RFC PR. Accepting the PR is interpreted as a yes vote while a request for changes is interpreted as a no vote. + +##### Acceptance + +If the end date of the vote arrives without a no vote from a member with binding vote, the RFC is accepted. It will be merged into the repo and assigned a number. Implementation can begin. + +##### Rejection + +If a single no vote is cast before the arrival of the end date the RFC is immediately rejected. The same RFC may be brought to a vote again in the future assuming the concerns that lead to the no vote are addressed. + +# How it Works +[how-it-works]: #how-it-works + +### Amending the RFC Process + +The RFC process should be amended through the RFC process. However, the TOC reserves the right to change the process via a super-majority vote in the unlikely even that the process prove so irreparably flawed as to preclude its amendment via the process. + +# Migration +[migration]: #migration + +This RFC should be accepted via the existing RFC process. The new process will take effect immediately after its ratification, superseding in its entirety the previous process. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? +- Why is this proposal the best? +- What is the impact of not doing this? + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. +Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. +This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. From 0999ea7b6f0414bf9bac2ca1bcebf48a797020a8 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 11 May 2022 18:09:43 -0400 Subject: [PATCH 071/372] Adds process details * Additional terminology around voting * Adds more details to the process section * What requires an RFC * Project vs Team RFCs Signed-off-by: Emily Casey --- text/0000-rfc-process.md | 96 ++++++++++++++++++++++++++++++---------- 1 file changed, 72 insertions(+), 24 deletions(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index fad9f4a8b..67c78d606 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -19,16 +19,17 @@ The RFC process: * ensures the motivation for a change is clear. * ensures the impact of a change on users is clear and migration path and backwards compatibility are considered. * aligns stakeholders on any changes to the user interface(s) and/or APIs (e.g. pack user interface, platform API, buildpack API). -* aligns stakeholders on any substantive architectural decision changes -* Aligns stakeholders on any processes or workflows adopted by the project. -* provides visibility to the community regarding on-going work and upcoming changes. +* aligns stakeholders on any substantive architectural changes. +* aligns stakeholders on any processes or workflows adopted by the project. +* provides visibility to the community regarding ongoing work and upcoming changes. * provides a mechanism by which any interested party can provide early feedback on an upcoming change. +* provides a version controlled record of our decisions and the thought-process behind them. * is open to anyone! We enthusiastically welcome RFCs from authors that have no formal role in project governance (yet ;p). The RFC process **is not**: -* A replacement for high-quality user-facing documentation. -* A transaction log of changes. Readers should be able to understand the change proposed without undue reference to previous RFCs. -* A feature request process. RFCs require a level of design and implementation detail that goes beyond a feature request. Pure feature requests should instead be initiated as discussions on the community repo, issues on this repo, or issues on component repos. These requests may serve as the motivation for future RFCs. +* a replacement for high-quality user-facing documentation(although high-quality RFCs enable the creation of high-quality documentation). +* a transaction log of changes. Readers should be able to understand the change proposed without undue reference to previous RFCs. +* a feature request process. RFCs require a level of design and implementation detail that goes beyond a feature request. Pure feature requests should instead be initiated as discussions on the community repo, issues on this repo, or issues on component repos. These requests may serve as the motivation for future RFCs. # Definitions @@ -44,12 +45,18 @@ The RFC process **is not**: **Project RFC**: An RFC with cross-cutting implications, requiring collaboration between multiple teams or affecting multiple personas. -**Team RFC**: An RFC with narrower implication in comparison to a project RFC, with work scoped to a single team and implications for a narrower set of personas. +**Team RFC**: An RFC with narrower implication in comparison to a project RFC, with work scoped to a single team and implications for a narrower set of personas. + +**Author**: The author or authors of an RFC are responsible for producing the draft RFC and updating it to incorporate feedback. Changes should not be made to a draft RFC without the author's consent. + +**Steward**: The steward of an RFC is responsible for shepherding an RFC through the process, including working with the author to ensure RFC completeness and quality, and building consensus among stakeholders. **Call for Votes**: When an RFC is deemed ready by a team lead, that team lead initiates the voting process with a call for votes. At this point the RFC is closed to modification. **End Date**: When a call for votes is initiated, an end date for voting is set. Any person wishing to vote on an RFC must do so by the end date. +**Voting Window**: The time period between a call for votes and the voting end date is referred to as the voting window. + **Lazy Consensus**: Voting on project level RFCs is done by lazy consensus. Any project member with a binding vote who has not voted by the end date is assumed to assent to the RFC. **Binding Vote**: Binding Votes have formal power within the RFC process. A single no vote by a project member with a binding vote is sufficient to reject an RFC. @@ -60,15 +67,41 @@ The RFC process **is not**: # Motivation [motivation]: #motivation -This RFC process is an evolution of our [previous process](0004-rfc-process.md). In comparison to the previous process this proposal makes two substantive cha +This RFC process is an evolution of our [previous process](0004-rfc-process.md). + +TODO # What it is [what-it-is]: #what-it-is ### What Types of Changes Require an RFC? +Any "substantive" change to the project requires an RFC. substantive includes but is not limited to: +* changes to the specification. +* the adoption, creation, or deprecation of a component (e.g. a new platform implementation, a new shared library, a new system buildpack). +* new features (e.g. a new pack command, a new flag on an existing pack command, an addition to the buildpack API) +* any major refactor that affects consumers of our libraries or materially impacts contribution. +* any major re-architecture especially if it has noteworthy implications for security or performance. +* introduction of new processes or changes to our existing processes including the RFC process. + +If there is any doubt, maintainers should prefer opening an "unecessary" RFC to surprising users was unexpectedly impactful changes. + #### Project vs Team RFCs +An RFC should be a project RFC if: +* it impacts the spec. +* it introduces a new component. +* its implementation necessitates coordination across multiple team. +* it meaningfully impacts multiple personas (e.g. buildpack authors and platform authors). +* the TOC requests that it be a project RFC. + +Given the nature of our project many RFCs will happen at the project level. However, some types of changes are more appropriately scoped to the team level. Some examples include: +* Platform example: additions to or modification of the pack CLI interface (e.g. [pack pull policy](0046-pack-pull-policy.md)) or library interface (e.g. [pack logging refactor](0002-pack-logging-refactor.md)), provided these changes do not require changes to components external to the platform team. +* BAT example: a new major version of the libcnb API. +* Implementation example: [Layer history](0102-history.md) or the lifecycle [multicall binary](0024-lifecycle-multicall-binary-for-build.md). +* Distribution example: [Update CNB Registry JSON Schema](https://github.com/buildpacks/rfcs/pull/45). +* Learning example: [intro video](0090-intro-video-script.md). + ### Process #### Drafting @@ -80,44 +113,56 @@ All RFCs begin life as a draft. Anyone wishing to propose a change to the projec - Submit a pull request. #### Finding a Steward -All RFCs, even project RFCs "belong" to a team. The team lead of the responsible team is the steward of the RFC. The author and the steward of an RFC may, at times, be the same person. +All RFCs, even project RFCs "belong" to a team. For project RFCs, The team lead of the responsible team is the steward of the RFC. For team RFCs any maintainer may be the steward. The author and the steward of an RFC may, at times, be the same person. -In many but not all cases the appropriate team to steward an RFC will be obvious. When the appropriate steward is not obvious the author should work with the community to find a home within one of the teams. Factors to consider when finding home include: +In many but not all cases the appropriate team to own an RFC will be obvious. When the appropriate team is not obvious, the author should work with the community to find a home within one of the teams. Factors to consider when finding home include: * Which team has the most relevant technical context? * Which team has the deepest understanding of the use-case and the needs of impacted personas? * Which team is responsible for the components most impacted by the proposed change? * Which team is enthusiastic about supporting the change? -If a home cannot be found for a draft RFC it remains in draft until one can be found. This does not necessary mean that the RFC is not a good idea or not something the project will take up eventually. It may simply be that the project does not have the bandwidth to prioritize this particular time. +If a home cannot be found for a draft RFC it remains in draft until one can be found. This does not necessary mean that the RFC is not a good idea or isn't something the project will take up eventually. It may simply be that the project does not have the bandwidth to prioritize this particular change at this particular time. #### Stewardship The steward and their team should: * work with the author of the RFC to ensure that the RFC is complete and implementable contingent upon approval. This can happen synchronously at team working groups or asynchronously via github and slack. - * raise visibility to and solicit feedback from other stakeholders including the TOC, other teams, and the community at large. This can happen synchronously at the project working group and asynchronously vai github and slack. + * raise visibility to and solicit feedback from other stakeholders including the TOC, other teams, and the community at large. This can happen synchronously at the project working group and asynchronously via github and slack. * drive consensus for the RFC by incorporating feedback from stakeholders so that the RFC has the best possible chance of approval during the voting process. - * ensure there is a plan in place to implement the RFC in a reasonable time frame, contingent upon approval. The team itself need not implement the RFC but we should not approve RFC for which we have no concrete plan to implement. + * ensure there is a plan in place to implement the RFC in a reasonable time frame, contingent upon approval. The team itself need not implement the RFC but we should not approve RFCs for which we have no concrete plan to implement. * in the case of complicated or risky RFCs, a POC should be developed at this stage to validate and de-risk the proposed design. #### Voting Ideally the voting should be a formality and not a moment to discover new disagreement, consensus already haven been driven by the steward. -When the steward deems the RFC ready and likely to be accepted they should formally call for votes and set an end date for voting. This process does not prescribe a length for the voting window, but stewards should make a good faith effort to ensure that all interested parties, and in particular those with binding votes, have adequate opportunity to read and cast votes. +When the steward deems the RFC ready and likely to be accepted they should formally call for votes and set an end date for voting. This process does not prescribe a length for the voting window, but stewards should make a good faith effort to ensure that all interested parties, and in particular those with binding votes, have adequate opportunity to review the finalized RFC and cast votes. + +The RFC may not be edited during the voting window. For project RFCs, all TOC members and all team leads are given a binding vote. For team RFCs, all team maintainers are given a binding vote. -Votes are cast via reviews on the RFC PR. Accepting the PR is interpreted as a yes vote while a request for changes is interpreted as a no vote. +Votes are cast via reviews on the RFC PR. Accepting the PR signifies a yes vote while a request for changes signifies a no vote. If all members with a binding vote vote in the affirmative, the voting window may close early. ##### Acceptance -If the end date of the vote arrives without a no vote from a member with binding vote, the RFC is accepted. It will be merged into the repo and assigned a number. Implementation can begin. +If the end date of the vote arrives without a no vote from a member with binding vote, the RFC is accepted. It will be merged into the repo and assigned a number. The steward should create a tracking issue to coordinate implementation and link the tracking issue in the RFC header. When this is complete, implementation can begin. ##### Rejection -If a single no vote is cast before the arrival of the end date the RFC is immediately rejected. The same RFC may be brought to a vote again in the future assuming the concerns that lead to the no vote are addressed. +If a single no vote is cast before the arrival of the end date the RFC is immediately rejected and the PR should be closed. + +The same RFC may be re-opened and brought to a vote again in the future assuming the concerns that lead to the no vote are addressed. + +### Amending an RFC + +TODO + +### Amending the RFC Process + +The RFC process should be amended through the RFC process. However, the TOC reserves the right to change the process via a super-majority vote in the unlikely even that the process prove so irreparably flawed as to preclude its amendment via the process. # How it Works [how-it-works]: #how-it-works @@ -136,6 +181,8 @@ This RFC should be accepted via the existing RFC process. The new process will t Why should we *not* do this? +TODO + # Alternatives [alternatives]: #alternatives @@ -143,20 +190,21 @@ Why should we *not* do this? - Why is this proposal the best? - What is the impact of not doing this? +TODO + # Prior Art [prior-art]: #prior-art -Discuss prior art, both the good and bad. +TODO + +* Borrows heavily from TEP and KEP processes # Unresolved Questions [unresolved-questions]: #unresolved-questions -- What parts of the design do you expect to be resolved before this gets merged? -- What parts of the design do you expect to be resolved through implementation of the feature? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? +TODO # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes -Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. -Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. -This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. + +N/A From 5154bfec42d4657638ac571d2b752b2665357156 Mon Sep 17 00:00:00 2001 From: Andrew Gracey Date: Wed, 18 May 2022 07:42:04 -0700 Subject: [PATCH 072/372] initial take --- text/0000-additonal-oci-artifacts.md | 98 ++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 text/0000-additonal-oci-artifacts.md diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md new file mode 100644 index 000000000..5e6a03377 --- /dev/null +++ b/text/0000-additonal-oci-artifacts.md @@ -0,0 +1,98 @@ +# Meta +[meta]: #meta +- Name: Additional OCI Artifacts from Buildpacks +- Start Date: 2022-04-24 +- Author(s): agracey +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +Allow buildpack authors to specify additional artifacts from their buildpacks to be uploaded to a registry along with the standard set of layers. + + +# Definitions +[definitions]: #definitions + +Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with. + +- WebAssembly (WASM) -- + +# Motivation +[motivation]: #motivation + +This allows for quiet a bit of additional flexibility for buildpack authors and platforms that are building on top of buildpacks. The two most obvious use cases are WebAssembly and Helm. + +For the first, It would be really cool for a buildpack to be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or a custom plugin system). + +Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, etc...) + + +# What it is +[what-it-is]: #what-it-is + +This provides a high level overview of the feature. + +- Define any new terminology. +- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor. +- Explaining the feature largely in terms of examples. +- If applicable, provide sample error messages, deprecation warnings, or migration guidance. +- If applicable, describe the differences between teaching this to existing users and new users. + +# How it Works +[how-it-works]: #how-it-works + +Add a field in the `layer.toml` spec called `additionalArtifact` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. + +For the WASM usecase, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following layer.toml: + +``` +[layer] +publish=false +cache=false + +[additionalArtifact] +tag=wasm +file=mywasmprogram.wasm + +[additionalArtifact.labels] +myLabel1=blah1-${build.tag} +myLabel2=blah2 +``` + + +# Migration +[migration]: #migration + +This functionality should be purely additive and not break existing buildpacks. + +# Drawbacks +[drawbacks]: #drawbacks + +This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. I think this can be mitigated with good docs and platform output. + +# Alternatives +[alternatives]: #alternatives + +Other ways of doing this would be to add the artifacts to the primary OCI image directly then pull back out in a future stage. This has two obvious issues: artifact size and escalation in tooling needed in pipeline. + +# Prior Art +[prior-art]: #prior-art + +None that I'm aware of. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- [] What to do when there are collisions in tags. +- [] Could the [Bindle](https://github.com/deislabs/bindle) project help this? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +The main addition to the spec is the addition of the fields in layer.toml as well as the changes to lifecycle to see them. + From 78c0b1f09b00719e6d8a03735c6697d828ca2094 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 25 Jan 2022 12:10:20 -0600 Subject: [PATCH 073/372] Copy template Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 80 ++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 text/0000-dot-profile-utility-buildpack.md diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md new file mode 100644 index 000000000..b4256f4e7 --- /dev/null +++ b/text/0000-dot-profile-utility-buildpack.md @@ -0,0 +1,80 @@ +# Meta +[meta]: #meta +- Name: (fill in the feature name: My Feature) +- Start Date: (fill in today's date: YYYY-MM-DD) +- Author(s): (Github usernames) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +One paragraph explanation of the feature. + +# Definitions +[definitions]: #definitions + +Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with. + +# Motivation +[motivation]: #motivation + +- Why should we do this? +- What use cases does it support? +- What is the expected outcome? + +# What it is +[what-it-is]: #what-it-is + +This provides a high level overview of the feature. + +- Define any new terminology. +- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor. +- Explaining the feature largely in terms of examples. +- If applicable, provide sample error messages, deprecation warnings, or migration guidance. +- If applicable, describe the differences between teaching this to existing users and new users. + +# How it Works +[how-it-works]: #how-it-works + +This is the technical portion of the RFC, where you explain the design in sufficient detail. + +The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. + +# Migration +[migration]: #migration + +This section should document breaks to public API and breaks in compatibility due to this RFC's proposed changes. In addition, it should document the proposed steps that one would need to take to work through these changes. Care should be give to include all applicable personas, such as platform developers, buildpack developers, buildpack users and consumers of buildpack images. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? +- Why is this proposal the best? +- What is the impact of not doing this? + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. +Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. +This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. From 2d22c3aac67c12db2befe2b56dd128608d80b173 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 25 Jan 2022 16:41:43 -0600 Subject: [PATCH 074/372] First draft of .profile utility buildpack RFC Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 93 +++++++++++++++------- 1 file changed, 66 insertions(+), 27 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index b4256f4e7..c7c896e92 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -1,65 +1,105 @@ # Meta [meta]: #meta -- Name: (fill in the feature name: My Feature) -- Start Date: (fill in today's date: YYYY-MM-DD) -- Author(s): (Github usernames) +- Name: `.profile` Utility Buildpack +- Start Date: 2022-01-25 +- Author(s): mboldt - Status: Draft - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) +- Supersedes: N/A # Summary [summary]: #summary -One paragraph explanation of the feature. +As part of RFC 93, [`.profile` scripts will cease to be supported by the platform API](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md#appprofile). +This RFC proposes developing a [utility buildpack](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md) to support `.profile` scripts to prevent regressions after RFC 93 is implemented. # Definitions [definitions]: #definitions -Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with. +*utility buildpack*: A buildpack officially supported by the Buildpack Authors' Tooling Team per [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md). # Motivation [motivation]: #motivation -- Why should we do this? -- What use cases does it support? -- What is the expected outcome? +[RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md) resolves to remove shell-specific logic from the CNB Specification. +Part of this includes removing support for `.profile` script in a future version of the Platform API. +RFC 93 recommends supporting the `.profile` script functionality in a utility buildpack to avoid regressions. +This proposal is to develop and support the `.profile` utility buildpack, allowing RFC 93 to be implemented without regression. # What it is [what-it-is]: #what-it-is -This provides a high level overview of the feature. +The target persona is a platofrm operator or implementor who wants to update to the latest platform API, while maintaining the `.profile` functionality for application developers. + +We propose developing and supporting a buildpack to provide an identical interface to the existing `.profile` functionality. +It will: + +- Detect a `.profile` file in the app dir +- Wrap the `.profile` file so that it implements the `exec.d` interface +- Add the `exec.d` executable to the `/exec.d` directory so the launcher will apply it + +## Example 1 (environment variables) + +Here is an example of a `.profile` script, inspired by paketo-buidpacks/node-engine: + +``` +memory_in_bytes="$(cat /sys/fs/cgroup/memory/memory.limit_in_bytes)" +MEMORY_AVAILABLE="$(( $memory_in_bytes / ( 1024 * 1024 ) ))" +export MEMORY_AVAILABLE +``` + +The wrapper should ensure that the `MEMORY_AVAILABLE` environment variable is set in the environment with the proper value. + +## Example 2 (file system side effects) + +With this contrived `.profile` script: + +``` +echo 'hello' >> "$HOME/hello" +``` + +The wrapper would not need to set any environment varibales, but should maintain the side effect of creating the `$HOME/hello` file. -- Define any new terminology. -- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor. -- Explaining the feature largely in terms of examples. -- If applicable, provide sample error messages, deprecation warnings, or migration guidance. -- If applicable, describe the differences between teaching this to existing users and new users. # How it Works [how-it-works]: #how-it-works -This is the technical portion of the RFC, where you explain the design in sufficient detail. +In short, the wrapper should do: + +``` +#!/bin/bash +set -eo +source .profile +env >&3 +``` -The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. +This will write the environment, including any variables set in `.profile`, to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). +Since it also executes the `.profile` script, any side effects will happen. +This will solve for both of the simple examples above. # Migration [migration]: #migration -This section should document breaks to public API and breaks in compatibility due to this RFC's proposed changes. In addition, it should document the proposed steps that one would need to take to work through these changes. Care should be give to include all applicable personas, such as platform developers, buildpack developers, buildpack users and consumers of buildpack images. +This buildpack is net new, so has no inherent migration considerations. + +For the realted migration concerns for removing shell functionality in general, see the [Migration Path section of RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md#migration-path). # Drawbacks [drawbacks]: #drawbacks -Why should we *not* do this? +This is a new component to maintain and support. +That being said, this will enable spec and lifecycle simplifications noted in [RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md) (where this approach was first suggested). +Also, [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md) resolves to support and maintain utility buidpacks. # Alternatives [alternatives]: #alternatives +If we do nothing, we introduce a regression in functionality, and force application developers to rework their `.profile` scripts. + - What other designs have been considered? - Why is this proposal the best? -- What is the impact of not doing this? # Prior Art [prior-art]: #prior-art @@ -69,12 +109,11 @@ Discuss prior art, both the good and bad. # Unresolved Questions [unresolved-questions]: #unresolved-questions -- What parts of the design do you expect to be resolved before this gets merged? -- What parts of the design do you expect to be resolved through implementation of the feature? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? +- Are there things that `.profile` scripts do that will not be covered by the exec.d interface? + For example, defining environment variables and side effects like writing files should be supported. + But, something like defining a bash function will not be supported. -# Spec. Changes (OPTIONAL) +# Spec. Changes [spec-changes]: #spec-changes -Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. -Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. -This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. + +None. From 5fe7c35c7c16c72bd687fbf0b2fc4f739d659e1e Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 26 Jan 2022 11:29:16 -0600 Subject: [PATCH 075/372] Add a note about using the same approach on Windows Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index c7c896e92..bc4e7d626 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -79,6 +79,13 @@ This will write the environment, including any variables set in `.profile`, to t Since it also executes the `.profile` script, any side effects will happen. This will solve for both of the simple examples above. +On Windows, for `.profile.bat` scripts, we can take the same approach of wrapping the script like: + +``` +call .profile.bat +set >&%CNB_EXEC_D_HANDLE% +``` + # Migration [migration]: #migration From 9a611585c30501584591e4c317409c8ed88d898e Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 26 Jan 2022 18:08:03 -0600 Subject: [PATCH 076/372] Fix typo in bash script snippet Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index bc4e7d626..f4ba3d333 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -70,7 +70,7 @@ In short, the wrapper should do: ``` #!/bin/bash -set -eo +set -euo pipefail source .profile env >&3 ``` From 006c244a9afc5c7937d00eb5e961883b31094dd7 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 27 Jan 2022 08:53:21 -0600 Subject: [PATCH 077/372] Add proposed buildpack ID buildpacksio/profile Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index f4ba3d333..8d25bf259 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -40,6 +40,8 @@ It will: - Wrap the `.profile` file so that it implements the `exec.d` interface - Add the `exec.d` executable to the `/exec.d` directory so the launcher will apply it +We propose the ID `buildpacksio/profie` for this buildpack (cf. [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md#what-it-is)). + ## Example 1 (environment variables) Here is an example of a `.profile` script, inspired by paketo-buidpacks/node-engine: From bfd67cfe1a4264a699d97780bd7b7fa0adb2c900 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 27 Jan 2022 09:00:52 -0600 Subject: [PATCH 078/372] Fix typos Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 8d25bf259..2a3026738 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -31,7 +31,7 @@ This proposal is to develop and support the `.profile` utility buildpack, allowi # What it is [what-it-is]: #what-it-is -The target persona is a platofrm operator or implementor who wants to update to the latest platform API, while maintaining the `.profile` functionality for application developers. +The target persona is a platform operator or implementor who wants to update to the latest platform API, while maintaining the `.profile` functionality for application developers. We propose developing and supporting a buildpack to provide an identical interface to the existing `.profile` functionality. It will: @@ -93,7 +93,7 @@ set >&%CNB_EXEC_D_HANDLE% This buildpack is net new, so has no inherent migration considerations. -For the realted migration concerns for removing shell functionality in general, see the [Migration Path section of RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md#migration-path). +For the related migration concerns for removing shell functionality in general, see the [Migration Path section of RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md#migration-path). # Drawbacks [drawbacks]: #drawbacks From 12ecd9b72f6577fbc82f83c33b78c9cc7d994c6c Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 27 Jan 2022 09:01:11 -0600 Subject: [PATCH 079/372] Add note about TOML formatting Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 2a3026738..83c00263a 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -77,6 +77,9 @@ source .profile env >&3 ``` +We acknoledge that the `env` output will need to be properly quoted to be valid TOML. +We will use or create a tool to handle that formatting. + This will write the environment, including any variables set in `.profile`, to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). Since it also executes the `.profile` script, any side effects will happen. This will solve for both of the simple examples above. From cff5b56829ae6d0f928984ad051eec70c0049647 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:56:07 -0600 Subject: [PATCH 080/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 83c00263a..c8a10db63 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -56,7 +56,7 @@ The wrapper should ensure that the `MEMORY_AVAILABLE` environment variable is se ## Example 2 (file system side effects) -With this contrived `.profile` script: +With this `.profile` script: ``` echo 'hello' >> "$HOME/hello" From 583035aa95b3f4259b4b1bb8407b9554834b15f4 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:56:17 -0600 Subject: [PATCH 081/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index c8a10db63..df3d3a8f7 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -116,7 +116,7 @@ If we do nothing, we introduce a regression in functionality, and force applicat # Prior Art [prior-art]: #prior-art -Discuss prior art, both the good and bad. +- # Unresolved Questions [unresolved-questions]: #unresolved-questions From 91a4fca7bef95bf71d16753b51fffac63e134b98 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:56:34 -0600 Subject: [PATCH 082/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index df3d3a8f7..1f62aaba3 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -18,7 +18,7 @@ This RFC proposes developing a [utility buildpack](https://github.com/buildpacks # Definitions [definitions]: #definitions -*utility buildpack*: A buildpack officially supported by the Buildpack Authors' Tooling Team per [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md). +*utility buildpack*: A buildpack officially supported by the Buildpack Authors' Tooling Team per [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md) that provides a generic capability. # Motivation [motivation]: #motivation From 6e41672d458f4200e9dbcbca1d192f4bd671d2c4 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:56:42 -0600 Subject: [PATCH 083/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 1f62aaba3..0a77ead18 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -103,7 +103,7 @@ For the related migration concerns for removing shell functionality in general, This is a new component to maintain and support. That being said, this will enable spec and lifecycle simplifications noted in [RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md) (where this approach was first suggested). -Also, [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md) resolves to support and maintain utility buidpacks. +Also, [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md) resolves to support and maintain utility buildpacks. # Alternatives [alternatives]: #alternatives From 4830d50678b4bbff346a98925498351c776c1e4a Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:56:48 -0600 Subject: [PATCH 084/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 0a77ead18..81fd9b1e3 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -110,8 +110,6 @@ Also, [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-u If we do nothing, we introduce a regression in functionality, and force application developers to rework their `.profile` scripts. -- What other designs have been considered? -- Why is this proposal the best? # Prior Art [prior-art]: #prior-art From ab806e76f3500030f8c1841499e5f8784dabfb30 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:56:56 -0600 Subject: [PATCH 085/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 81fd9b1e3..2fb4717ca 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -44,7 +44,7 @@ We propose the ID `buildpacksio/profie` for this buildpack (cf. [RFC 97](https:/ ## Example 1 (environment variables) -Here is an example of a `.profile` script, inspired by paketo-buidpacks/node-engine: +Here is an example of a `.profile` script, inspired by paketo-buildpacks/node-engine: ``` memory_in_bytes="$(cat /sys/fs/cgroup/memory/memory.limit_in_bytes)" From 2fc9094db01548b259910e47e61fc70039803b31 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:57:02 -0600 Subject: [PATCH 086/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Anthony Emengo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 2fb4717ca..f11333d77 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -77,7 +77,7 @@ source .profile env >&3 ``` -We acknoledge that the `env` output will need to be properly quoted to be valid TOML. +We acknowledge that the `env` output will need to be properly quoted to be valid TOML. We will use or create a tool to handle that formatting. This will write the environment, including any variables set in `.profile`, to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). From a66e2038afc5d51703ad1bcf16fb77b67bb7ef87 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Wed, 2 Feb 2022 07:58:45 -0600 Subject: [PATCH 087/372] Minor edit Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index f11333d77..7207c272e 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -94,7 +94,7 @@ set >&%CNB_EXEC_D_HANDLE% # Migration [migration]: #migration -This buildpack is net new, so has no inherent migration considerations. +This buildpack is new, so has no inherent migration considerations. For the related migration concerns for removing shell functionality in general, see the [Migration Path section of RFC 93](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md#migration-path). From bb1d8d7f4b04528df6d1defe48816debd6357e8f Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 3 Feb 2022 07:32:23 -0600 Subject: [PATCH 088/372] Update text/0000-dot-profile-utility-buildpack.md Co-authored-by: Joe Kutner Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 7207c272e..a8e3a86bb 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -40,7 +40,7 @@ It will: - Wrap the `.profile` file so that it implements the `exec.d` interface - Add the `exec.d` executable to the `/exec.d` directory so the launcher will apply it -We propose the ID `buildpacksio/profie` for this buildpack (cf. [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md#what-it-is)). +We propose the ID `buildpacksio/profile` for this buildpack (cf. [RFC 97](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md#what-it-is)). ## Example 1 (environment variables) From 46d0668bf62a722af533c425ea16a13c4ad7cd81 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 3 Feb 2022 09:07:08 -0600 Subject: [PATCH 089/372] Add shell version compatibility Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index a8e3a86bb..63ebf20ee 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -91,6 +91,8 @@ call .profile.bat set >&%CNB_EXEC_D_HANDLE% ``` +Per the [Operating System Conventions in the CNB spec](https://github.com/buildpacks/spec#operating-system-conventions), this buildpack will support scripts compatible with bash version 3 or greater on Linux, and cmd.exe on Windows. + # Migration [migration]: #migration From 1e1942a13e7adb77a8706bc2770256eaa188984d Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 3 Feb 2022 09:08:15 -0600 Subject: [PATCH 090/372] Fix typo Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 63ebf20ee..454c78271 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -62,7 +62,7 @@ With this `.profile` script: echo 'hello' >> "$HOME/hello" ``` -The wrapper would not need to set any environment varibales, but should maintain the side effect of creating the `$HOME/hello` file. +The wrapper would not need to set any environment variables, but should maintain the side effect of creating the `$HOME/hello` file. # How it Works From 60f6ead35d2328803c9a45f4d610c0e679433ec4 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 3 Feb 2022 09:08:39 -0600 Subject: [PATCH 091/372] Add prior art Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 454c78271..71ed99811 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -116,7 +116,18 @@ If we do nothing, we introduce a regression in functionality, and force applicat # Prior Art [prior-art]: #prior-art -- +- Current CNB spec: + - `.profile` scripts are part of the buildpack app interface ([spec](https://github.com/buildpacks/spec/blob/main/buildpack.md#app-interface)) + - `.profile.d/` scripts are part of the buildpack build phase output ([spec](https://github.com/buildpacks/spec/blob/main/buildpack.md#build)). + +- Profile scripts: + - Heroku supports [`.profile` scripts](https://devcenter.heroku.com/articles/dynos#the-profile-file) and [`.profile.d/` scripts](https://devcenter.heroku.com/articles/buildpack-api#profile-d-scripts). + - Cloud Foundry supports [`.profile and .profile.d/ scripts](https://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html#profile). + +- Prior related RFCs: + - [Officially Supported Utility Buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md) + - [Remove Shell Processes](https://github.com/buildpacks/rfcs/blob/main/text/0093-remove-shell-processes.md) + - [Exec.d - Shell-Free Profile.d](https://github.com/buildpacks/rfcs/blob/main/text/0057-exec.d-shell-free-profile-d.md) # Unresolved Questions [unresolved-questions]: #unresolved-questions From 00bd1b4f390316bd66b6e69069c7cdbe3bd6f55b Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 3 Feb 2022 09:10:52 -0600 Subject: [PATCH 092/372] Add open question about run image without bash Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 71ed99811..278e0c661 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -136,6 +136,9 @@ If we do nothing, we introduce a regression in functionality, and force applicat For example, defining environment variables and side effects like writing files should be supported. But, something like defining a bash function will not be supported. +- What if the run image does not contain bash (e.g., `FROM scratch`)? + Should it error, no-op, should this buildpack add a layer with bash? + # Spec. Changes [spec-changes]: #spec-changes From 36faa095fbdc6f2888d1d03a8be7e7b6616bd793 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 8 Feb 2022 07:57:21 -0600 Subject: [PATCH 093/372] Address what happens if bash is not in the run image Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 278e0c661..4edc29259 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -132,13 +132,20 @@ If we do nothing, we introduce a regression in functionality, and force applicat # Unresolved Questions [unresolved-questions]: #unresolved-questions -- Are there things that `.profile` scripts do that will not be covered by the exec.d interface? +- **Are there things that `.profile` scripts do that will not be covered by the exec.d interface?** For example, defining environment variables and side effects like writing files should be supported. But, something like defining a bash function will not be supported. -- What if the run image does not contain bash (e.g., `FROM scratch`)? +- **What if the run image does not contain bash (e.g., `FROM scratch`)?** Should it error, no-op, should this buildpack add a layer with bash? + Per discussion, we will document that this buildpack requires bash to be in the run image, and this buildpack will not add bash. + If bash is not in the run image, the exec.d wrapper will error out. + + We may consider developing another buildpack to install bash in the future, but that is out of scope for this RFC. + We have done a proof-of-concept of [running a statically linked bash on scratch](https://github.com/mboldt/scratch-play/tree/main/bash), using the [bash-static Debian package](https://packages.debian.org/unstable/bash-static). + We have also found some prior art if we need to [compile our own static bash](https://github.com/robxu9/bash-static). + # Spec. Changes [spec-changes]: #spec-changes From 69e36a682e102e404e1a2d470e6ba2ed1d9e20a8 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 8 Feb 2022 08:22:41 -0600 Subject: [PATCH 094/372] Cite the "no output image dependecy" clause in utility buildpack RFC Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 4edc29259..7a9f81d8a 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -139,12 +139,16 @@ If we do nothing, we introduce a regression in functionality, and force applicat - **What if the run image does not contain bash (e.g., `FROM scratch`)?** Should it error, no-op, should this buildpack add a layer with bash? - Per discussion, we will document that this buildpack requires bash to be in the run image, and this buildpack will not add bash. - If bash is not in the run image, the exec.d wrapper will error out. + Note that the [utility buildpacks RFC](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md#how-it-works) states: + > The buildpack MUST NOT install any dependencies in the output image + which precludes this buildpack from adding bash to the run image. - We may consider developing another buildpack to install bash in the future, but that is out of scope for this RFC. + That said, if another person or project creates a bash buildpack, it could be combined with this buildpack. We have done a proof-of-concept of [running a statically linked bash on scratch](https://github.com/mboldt/scratch-play/tree/main/bash), using the [bash-static Debian package](https://packages.debian.org/unstable/bash-static). - We have also found some prior art if we need to [compile our own static bash](https://github.com/robxu9/bash-static). + We have also found some prior art about [creating a statically linked bash](https://github.com/robxu9/bash-static). + + We will document that this buildpack requires bash to be in the run image, and this buildpack will not add bash. + If bash is not in the run image, the exec.d wrapper will error out. # Spec. Changes [spec-changes]: #spec-changes From 35a44731d6fb83b510879e904a0de717b6e01a45 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 8 Mar 2022 10:14:19 -0600 Subject: [PATCH 095/372] Be more precise about the env var output format Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 7a9f81d8a..795e60053 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -68,27 +68,29 @@ The wrapper would not need to set any environment variables, but should maintain # How it Works [how-it-works]: #how-it-works -In short, the wrapper should do: +The `.profile` wrapper will: + +1. Source the `.profile` script. +1. Write the environment variables to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). + +We will write a small program, `write-env-toml`, to write all environment variables in the `exec.d` output TOML format. + ``` #!/bin/bash -set -euo pipefail source .profile -env >&3 +write-env-toml >&3 ``` -We acknowledge that the `env` output will need to be properly quoted to be valid TOML. -We will use or create a tool to handle that formatting. - -This will write the environment, including any variables set in `.profile`, to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). -Since it also executes the `.profile` script, any side effects will happen. -This will solve for both of the simple examples above. +Sourcing the `.profile` script will execute any side effects. +Writing the `exec.d` output TOML will ensure that all environment variables will be set. +So, this will solve for both of the simple examples above. On Windows, for `.profile.bat` scripts, we can take the same approach of wrapping the script like: ``` call .profile.bat -set >&%CNB_EXEC_D_HANDLE% +write-env-toml >&%CNB_EXEC_D_HANDLE% ``` Per the [Operating System Conventions in the CNB spec](https://github.com/buildpacks/spec#operating-system-conventions), this buildpack will support scripts compatible with bash version 3 or greater on Linux, and cmd.exe on Windows. From 5903145d4c2d3a1e94f83b90f85ea144dde303f5 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 8 Mar 2022 10:30:26 -0600 Subject: [PATCH 096/372] Describe the two programs that will implement the buildpack Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 32 ++++++++++------------ 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 795e60053..bc53650cf 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -68,31 +68,29 @@ The wrapper would not need to set any environment variables, but should maintain # How it Works [how-it-works]: #how-it-works -The `.profile` wrapper will: +We will write two small programs to implement the buildpack. -1. Source the `.profile` script. -1. Write the environment variables to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). +- `dot-profile-wrapper` will be the main entry point. +- `write-env-toml` will write all environment variables to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). -We will write a small program, `write-env-toml`, to write all environment variables in the `exec.d` output TOML format. +On Linux, `dot-profile-wrapper` will: +1. Ensure `bash` exists on the run image, and error if it does not. +1. Launch a `bash` shell to: + 1. Source the `.profile` script, which may set environment variables and/or have other side effects. + 1. Call `write-env-toml` to capture the environment variables. -``` -#!/bin/bash -source .profile -write-env-toml >&3 -``` +Likewise on Windows, `dot-profile-wrapper` will: -Sourcing the `.profile` script will execute any side effects. +1. Ensure `cmd.exe` exists on the run image, and error if it does not. +1. Launch a `cmd.exe` shell to: + 1. Source the `.profile.bat` script, which may set environment variables and/or have other side effects. + 1. Call `write-env-toml` to capture the environment variables. + +Sourcing the `.profile`/`.profile.bat` script will execute any side effects. Writing the `exec.d` output TOML will ensure that all environment variables will be set. So, this will solve for both of the simple examples above. -On Windows, for `.profile.bat` scripts, we can take the same approach of wrapping the script like: - -``` -call .profile.bat -write-env-toml >&%CNB_EXEC_D_HANDLE% -``` - Per the [Operating System Conventions in the CNB spec](https://github.com/buildpacks/spec#operating-system-conventions), this buildpack will support scripts compatible with bash version 3 or greater on Linux, and cmd.exe on Windows. # Migration From e3c62ea98bb1b69c2d99d99e6d20cff599d5f359 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 8 Mar 2022 10:43:12 -0600 Subject: [PATCH 097/372] Discourage a bash buildpack in favor of more secure run images Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index bc53650cf..dc3cfbf98 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -141,11 +141,9 @@ If we do nothing, we introduce a regression in functionality, and force applicat Note that the [utility buildpacks RFC](https://github.com/buildpacks/rfcs/blob/main/text/0097-official-utility-buildpacks.md#how-it-works) states: > The buildpack MUST NOT install any dependencies in the output image - which precludes this buildpack from adding bash to the run image. + which precludes this buildpack (or any CNB utility buildpack) from adding bash to the run image. - That said, if another person or project creates a bash buildpack, it could be combined with this buildpack. - We have done a proof-of-concept of [running a statically linked bash on scratch](https://github.com/mboldt/scratch-play/tree/main/bash), using the [bash-static Debian package](https://packages.debian.org/unstable/bash-static). - We have also found some prior art about [creating a statically linked bash](https://github.com/robxu9/bash-static). + While one can imagine a buildpack to add bash, we would like to see the community move to more secure run images that don't include bash. We will document that this buildpack requires bash to be in the run image, and this buildpack will not add bash. If bash is not in the run image, the exec.d wrapper will error out. From 0ab925e6b3a3524ad2f84bd847b8cbd9a0d0e239 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Tue, 8 Mar 2022 11:02:35 -0600 Subject: [PATCH 098/372] Color the discussion around .profile defining bash functions for app Co-authored-by: Emily Casey Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index dc3cfbf98..45500f324 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -136,6 +136,13 @@ If we do nothing, we introduce a regression in functionality, and force applicat For example, defining environment variables and side effects like writing files should be supported. But, something like defining a bash function will not be supported. + This is definitely an edge case. + The app profile runs after the buildpack provided profiles and just before the process type is executed so the only thing that could depend on one of the functions in the process. + This will almost certainly never happen with a buildpack-provided process (the buildpack depending on the app to define a bash function would be very weird and brittle). + + If the user provides the process definition (e.g. with a procfile) this is slightly more plausible. + But in this case there is a very easy workaround - source a script containing the functions as part of the command provided in the procfile. + - **What if the run image does not contain bash (e.g., `FROM scratch`)?** Should it error, no-op, should this buildpack add a layer with bash? From af553ce817712f7e7105c81bb5bb216e8d5b3fa8 Mon Sep 17 00:00:00 2001 From: Mikey Boldt Date: Thu, 24 Mar 2022 13:00:24 -0500 Subject: [PATCH 099/372] Leave implementation details to the BAT team post-RFC approval Signed-off-by: Mikey Boldt --- text/0000-dot-profile-utility-buildpack.md | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0000-dot-profile-utility-buildpack.md index 45500f324..e5125da48 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0000-dot-profile-utility-buildpack.md @@ -68,24 +68,19 @@ The wrapper would not need to set any environment variables, but should maintain # How it Works [how-it-works]: #how-it-works -We will write two small programs to implement the buildpack. - -- `dot-profile-wrapper` will be the main entry point. -- `write-env-toml` will write all environment variables to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). - -On Linux, `dot-profile-wrapper` will: +On Linux, the wrapper will: 1. Ensure `bash` exists on the run image, and error if it does not. 1. Launch a `bash` shell to: 1. Source the `.profile` script, which may set environment variables and/or have other side effects. - 1. Call `write-env-toml` to capture the environment variables. + 1. Write the environment variables to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). -Likewise on Windows, `dot-profile-wrapper` will: +Likewise on Windows, the wrapper will: 1. Ensure `cmd.exe` exists on the run image, and error if it does not. 1. Launch a `cmd.exe` shell to: 1. Source the `.profile.bat` script, which may set environment variables and/or have other side effects. - 1. Call `write-env-toml` to capture the environment variables. + 1. Write the environment variables to the [`exec.d` output TOML](https://github.com/buildpacks/spec/blob/main/buildpack.md#execd-output-toml). Sourcing the `.profile`/`.profile.bat` script will execute any side effects. Writing the `exec.d` output TOML will ensure that all environment variables will be set. @@ -155,6 +150,15 @@ If we do nothing, we introduce a regression in functionality, and force applicat We will document that this buildpack requires bash to be in the run image, and this buildpack will not add bash. If bash is not in the run image, the exec.d wrapper will error out. +- **How should we implement the buildpack?** + Should it be a bash script? + A Go binary (or two)? + What should we use for CI, frameworks, etc.? + + We will leave these implementation details to the Buildpack Authors' Tooling sub-team. + We can finalize these details after the RFC is approved. + We have captured some initial ideas for the [project plumbing](https://github.com/mboldt/utility-buildpack-plumbing) and had some discussion about [implementation details](https://github.com/buildpacks/rfcs/pull/200#discussion_r828063918) to seed those conversations. + # Spec. Changes [spec-changes]: #spec-changes From 5d06f94d729bd80669ca7fa9341ff87023df2b79 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Tue, 24 May 2022 20:08:39 +0100 Subject: [PATCH 100/372] RFC 0104 [#200] Signed-off-by: Sambhav Kothari --- ...y-buildpack.md => 0104-dot-profile-utility-buildpack.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-dot-profile-utility-buildpack.md => 0104-dot-profile-utility-buildpack.md} (98%) diff --git a/text/0000-dot-profile-utility-buildpack.md b/text/0104-dot-profile-utility-buildpack.md similarity index 98% rename from text/0000-dot-profile-utility-buildpack.md rename to text/0104-dot-profile-utility-buildpack.md index e5125da48..bdd60c2e5 100644 --- a/text/0000-dot-profile-utility-buildpack.md +++ b/text/0104-dot-profile-utility-buildpack.md @@ -3,10 +3,10 @@ - Name: `.profile` Utility Buildpack - Start Date: 2022-01-25 - Author(s): mboldt -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#200](https://github.com/buildpacks/rfcs/pull/200) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: N/A # Summary From 64fd0a9b6dac1cb9d6c567fd8b72f3457dcae065 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 25 May 2022 10:13:41 -0700 Subject: [PATCH 101/372] fix typo Signed-off-by: Terence Lee --- .../0066-lifecycle-prelease-version-and-experimental-section.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0066-lifecycle-prelease-version-and-experimental-section.md b/text/0066-lifecycle-prelease-version-and-experimental-section.md index 201e353c1..165ad4f41 100644 --- a/text/0066-lifecycle-prelease-version-and-experimental-section.md +++ b/text/0066-lifecycle-prelease-version-and-experimental-section.md @@ -37,7 +37,7 @@ A prerelease API is a non-finalized API but is in a testable state. The the API When Lifecycle supports a prerelease API, it can be treated like any other API version and be used in any mode: `experimental`, `supported`, `deprecated`. ### Experimental Features in the API -For APIs that will need to stabilize over a long time span, they can get added as an "experimental" section inside the API as **Experimental Features**. These will go out into official API releases. Using that part of the API will be experimental and susceptible to change in an upcoming release. This will require a bit more rigor, but will allow us to evolve the experimental sections overtime without as much pressure to "get it right" or "block a release". The downside is that the lifecycle will need to suppert these experimental features since they're part of the API. If the experimental API changes a lot, lifecycle will need to support these differences. +For APIs that will need to stabilize over a long time span, they can get added as an "experimental" section inside the API as **Experimental Features**. These will go out into official API releases. Using that part of the API will be experimental and susceptible to change in an upcoming release. This will require a bit more rigor, but will allow us to evolve the experimental sections overtime without as much pressure to "get it right" or "block a release". The downside is that the lifecycle will need to support these experimental features since they're part of the API. If the experimental API changes a lot, lifecycle will need to support these differences. There will be a new `CNB_PLATFORM_EXPERIMENTAL_FEATURES` environment variable to control ALL experimental features by the platform. This will let a platform decide if they want to turn on experimental features. For now, all features will be turned on/off for simplicity. If the need arises, a list in the future can be explored. From d5d34d4aa4cc132baf450c983e9c295be2f2b467 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 1 Jun 2022 09:58:41 -0400 Subject: [PATCH 102/372] Fix issue template It should be nested under .github Signed-off-by: Natalie Arellano --- {ISSUE_TEMPLATE => .github/ISSUE_TEMPLATE}/tracking.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename {ISSUE_TEMPLATE => .github/ISSUE_TEMPLATE}/tracking.md (100%) diff --git a/ISSUE_TEMPLATE/tracking.md b/.github/ISSUE_TEMPLATE/tracking.md similarity index 100% rename from ISSUE_TEMPLATE/tracking.md rename to .github/ISSUE_TEMPLATE/tracking.md From 541fb344ce1c33455e70ce6ec6d11afe56a686fb Mon Sep 17 00:00:00 2001 From: Andrew Gracey Date: Sun, 19 Jun 2022 11:17:29 -0700 Subject: [PATCH 103/372] Ready for review --- text/0000-additonal-oci-artifacts.md | 43 ++++++++++++---------------- 1 file changed, 18 insertions(+), 25 deletions(-) diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md index 5e6a03377..71140e541 100644 --- a/text/0000-additonal-oci-artifacts.md +++ b/text/0000-additonal-oci-artifacts.md @@ -14,54 +14,47 @@ Allow buildpack authors to specify additional artifacts from their buildpacks to be uploaded to a registry along with the standard set of layers. - # Definitions [definitions]: #definitions Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with. -- WebAssembly (WASM) -- +- WebAssembly (WASM) -- Portable binaries to be run with a WebAssembly virtual machine. +- OCI Artifact -- Any blob of data that can be stored in an OCI registry # Motivation [motivation]: #motivation -This allows for quiet a bit of additional flexibility for buildpack authors and platforms that are building on top of buildpacks. The two most obvious use cases are WebAssembly and Helm. - -For the first, It would be really cool for a buildpack to be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or a custom plugin system). - -Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, etc...) - +Cloud Native Buildpacks are awesome for building container images but come with a draw back that they can only produce a single output. This is limiting to platform builders because there are many cases where you might want to have multiple outputs from the same build process. # What it is [what-it-is]: #what-it-is -This provides a high level overview of the feature. +This change allows for additional flexibility in what buildpack authors and platforms can build on top of buildpacks. Two initial use cases are WebAssembly and Helm. + +For the first, a buildpack would be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or runwasi). -- Define any new terminology. -- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor. -- Explaining the feature largely in terms of examples. -- If applicable, provide sample error messages, deprecation warnings, or migration guidance. -- If applicable, describe the differences between teaching this to existing users and new users. +Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, ConfigMaps to pass around out-of-band, etc...) # How it Works [how-it-works]: #how-it-works Add a field in the `layer.toml` spec called `additionalArtifact` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. -For the WASM usecase, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following layer.toml: +For the WASM usecase, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following `layer.toml`: ``` -[layer] -publish=false -cache=false +[types] +publish = false +cache = false -[additionalArtifact] -tag=wasm -file=mywasmprogram.wasm +[[additionalArtifact]] +tag = "wasm" +file = "mywasmprogram.wasm" # Should be a file in $LAYERS_DIR -[additionalArtifact.labels] -myLabel1=blah1-${build.tag} -myLabel2=blah2 +[[additionalArtifact.labels]] +ConfigMediaType = "application/vnd.wasm.config.v1+json" +ContentLayerMediaType = "application/vnd.wasm.content.layer.v1+wasm" ``` @@ -89,7 +82,7 @@ None that I'm aware of. [unresolved-questions]: #unresolved-questions - [] What to do when there are collisions in tags. -- [] Could the [Bindle](https://github.com/deislabs/bindle) project help this? +- [] Could the [Bindle](https://github.com/deislabs/bindle) project help? # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From a2c06a553bf822fdd6d3aaf2e23139e79eaa4b96 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:32:53 -0400 Subject: [PATCH 104/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index f66f7c30f..800928f0a 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -33,10 +33,10 @@ For a given application, a build that uses extensions could be optimized by crea Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. The order of operations would be something like the following: * analyze * detect -* restore -* run extensions' bin/build, output Dockerfiles are written to a volume -* apply Dockerfiles to run image (could run in parallel with below two) +* run extensions' bin/generate, output Dockerfiles are written to a volume +* apply Dockerfiles to run image (could run in parallel with build image extension) * apply Dockerfiles to build image +* restore * build * export From 2935c8a3dd2dccbc1a28bd370e5b6ff3df74a9fc Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:33:03 -0400 Subject: [PATCH 105/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 800928f0a..0e6b8174a 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -44,7 +44,7 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap A builder image may include any number of "extensions" directories in `/cnb/ext/`. -Extensions are similar to buildpacks: they have a `/bin/build` and `/bin/detect` executable. +Extensions are similar to buildpacks: they have two executables: `/bin/detect` and `/bin/generate`. The interface for these executables is similar to a buildpack's `/bin/detect` and `/bin/build`. However, instead of a `buildpack.toml` file, extensions have a `extension.toml` file: ```toml api = "" From 01786f8ee6846acbfc73382e82809ffd770c4940 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:34:11 -0400 Subject: [PATCH 106/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 0e6b8174a..1000da8e8 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -73,6 +73,7 @@ Unlike buildpacks, Extensions participate in the buildpack detection process, with the same UID, GID, and interface for `/bin/detect`. However, - `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. +- If an extension is missing `/bin/detect`, the extension root is treated as a pre-populated output directory (i.e., extensions can include a static build plan). - Extensions may only output `provides` entries to the build plan. They must not output `requires`. - Extensions are not included in `order` definitions (e.g., in `builder.toml`); instead, a separate `order-ext` table should be used. The `order-ext` table will be prepended to the provided `order` (as if `order-ext` were a meta-buildpack). - Extensions are always `optional`. From 10a02f28f1907f85e536951e86a089afe987e1c1 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:34:24 -0400 Subject: [PATCH 107/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 1000da8e8..5aa41f09a 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -75,7 +75,7 @@ However, - `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. - If an extension is missing `/bin/detect`, the extension root is treated as a pre-populated output directory (i.e., extensions can include a static build plan). - Extensions may only output `provides` entries to the build plan. They must not output `requires`. -- Extensions are not included in `order` definitions (e.g., in `builder.toml`); instead, a separate `order-ext` table should be used. The `order-ext` table will be prepended to the provided `order` (as if `order-ext` were a meta-buildpack). +- Extensions are not included in `order` definitions (e.g., in `builder.toml`); instead, a separate `order-extensions` table should be used. The `order-extensions` table will be prepended to each group in the provided `order` (as if `order-extensions` were a composite buildpack). - Extensions are always `optional`. Extensions generate Dockerfiles before the regular buildpack build phase. From 2c7bb58fbfc4177e97ee8a33759a4f055674c988 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:34:30 -0400 Subject: [PATCH 108/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 5aa41f09a..6e0c8e176 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -81,7 +81,7 @@ However, Extensions generate Dockerfiles before the regular buildpack build phase. To generate these Dockerfiles, the lifecycle executes the extension's `/bin/build` executable with the same UID, GID, and interface as regular buildpacks. However, -- Extensions `/bin/build` must not write to the app directory. +- Extensions `/bin/generate` must not write to the app directory. - Extensions `/bin/build` may be executed in parallel. - Extensions `` directory is replaced by an `` directory. - If an extension is missing `/bin/build`, the extension root is treated as a pre-populated `` directory. From 2c6ca724788cf861ad5dd142a7b8e24efa3bb341 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:35:25 -0400 Subject: [PATCH 109/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 6e0c8e176..4588ea493 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -82,7 +82,6 @@ Extensions generate Dockerfiles before the regular buildpack build phase. To generate these Dockerfiles, the lifecycle executes the extension's `/bin/build` executable with the same UID, GID, and interface as regular buildpacks. However, - Extensions `/bin/generate` must not write to the app directory. -- Extensions `/bin/build` may be executed in parallel. - Extensions `` directory is replaced by an `` directory. - If an extension is missing `/bin/build`, the extension root is treated as a pre-populated `` directory. From aa74ace4c16fd53174bf12547f4733a4b3b46d03 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:35:34 -0400 Subject: [PATCH 110/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 4588ea493..9c13d95a7 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -99,7 +99,7 @@ Support for other instruction formats, e.g., LLB JSON files, could be added in t `build.Dockerfile`, `run.Dockerfile`, and `Dockerfile` target the builder image, runtime base image, or both base images, respectively. -If no Dockerfiles are present, `/bin/build` may still consume build plan entries and add metadata to `build.toml`/`launch.toml`. +If no Dockerfiles are present, `/bin/generate` may still consume build plan entries and add metadata to `build.toml`/`launch.toml`. Dockerfiles are applied to their corresponding base images after all extensions are executed and before any regular buildpacks are executed. Dockerfiles are applied in the order determined during buildpack detection. When multiple Dockerfiles are applied, the intermediate image generated from the application of the current Dockerfile will be provided as the `base_image` ARG to the next Dockerfile. Dockerfiles that target the run image (only) may ignore the provided `base_image` (e.g., `FROM some-other-image`). Dockerfiles that change the runtime base image may still use `COPY --from=${base_image}`. From 10b6c9fce0b12c2215f6ae5b8726afe846b5d72c Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:35:41 -0400 Subject: [PATCH 111/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 9c13d95a7..f0d15d83c 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -85,7 +85,7 @@ However, - Extensions `` directory is replaced by an `` directory. - If an extension is missing `/bin/build`, the extension root is treated as a pre-populated `` directory. -After `/bin/build` executes, the `` directory may contain +After `/bin/generate` executes, the `` directory may contain - `build.toml`, with the same contents as a normal buildpack's `build.toml`, but - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` or `Dockerfile` - `launch.toml`, with the same contents as a normal buildpack's `launch.toml`, but From 1cc27898aecba63d16bc0453a12068364281122e Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:37:20 -0400 Subject: [PATCH 112/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index f0d15d83c..c66584913 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -119,7 +119,7 @@ A runtime base image may indicate that it preserves ABI compatibility by adding This example extension would allow an app to provide runtime and build-time base image extensions as "run.Dockerfile" and "build.Dockerfile." The app developer can decide whether the extensions are rebasable. -##### `/cnb/ext/com.example.appext/bin/build` +##### `/cnb/ext/com.example.appext/bin/generate` ``` #!/bin/sh [ -f build.Dockerfile ] && cp build.Dockerfile "$1/" From a2282f17aab2cb817776a86e5ca1559269d4ea80 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:37:28 -0400 Subject: [PATCH 113/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index c66584913..1e5f11b95 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -132,7 +132,7 @@ This example extension would allow a builder to install RPMs for each language r Note: The Dockerfiles referenced must disable rebasing, and build times will be slower compared to buildpack-provided runtimes. -##### `/cnb/ext/com.example.rpmext/bin/build` +##### `/cnb/ext/com.example.rpmext/bin/generate` ``` #!/bin/sh [ -f Gemfile.lock ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-ruby" "$1/Dockerfile" From 9ba0bc23a42a34f9e68d87dc4be40d0d95e4fbe9 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:37:39 -0400 Subject: [PATCH 114/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 1e5f11b95..166da5510 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -144,7 +144,7 @@ Note: The Dockerfiles referenced must disable rebasing, and build times will be The same Dockerfile format may be used to create new base images or modify existing base images outside of the app build process (e.g., before creating a builder). Any specified labels override existing values. -Dockerfiles that are used to create a base image must create a `/cnb/image/genpkgs` executable that outputs a [CycloneDX](https://cyclonedx.org)-formatted list of packages in the image with PURL IDs when invoked. This executable is executed after all Dockerfiles are applied, and the output replaces the label `io.buildpacks.sbom`. This label doubles as a Software Bill-of-Materials for the base image. In the future, this label will serve as a starting point for the application SBoM. +The project will provide tooling that can be used to scan the extended run image. For more information, see https://github.com/buildpacks/rfcs/pull/195. ### Example Dockerfiles From 95336429e52113fb609c753dfe06b72a3416949e Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:37:52 -0400 Subject: [PATCH 115/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 166da5510..1e2223230 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -42,7 +42,7 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap ### Dynamically-applied Dockerfiles -A builder image may include any number of "extensions" directories in `/cnb/ext/`. +A builder image may include any number of "extensions" directories in `/cnb/extensions/`. Extensions are similar to buildpacks: they have two executables: `/bin/detect` and `/bin/generate`. The interface for these executables is similar to a buildpack's `/bin/detect` and `/bin/build`. However, instead of a `buildpack.toml` file, extensions have a `extension.toml` file: From bfc5781f99ad2ac4d3a301a4185db806f57b48cd Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:38:00 -0400 Subject: [PATCH 116/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 1e2223230..1825510c9 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -79,7 +79,7 @@ However, - Extensions are always `optional`. Extensions generate Dockerfiles before the regular buildpack build phase. -To generate these Dockerfiles, the lifecycle executes the extension's `/bin/build` executable with the same UID, GID, and interface as regular buildpacks. +To generate these Dockerfiles, the lifecycle executes the extension's `/bin/generate` executable with the same UID, GID, and interface as regular buildpacks. However, - Extensions `/bin/generate` must not write to the app directory. - Extensions `` directory is replaced by an `` directory. From 68773ac588014426e0a51f07230c94eb1b3eaced Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:38:32 -0400 Subject: [PATCH 117/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 1825510c9..11cf5d3dc 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -83,7 +83,7 @@ To generate these Dockerfiles, the lifecycle executes the extension's `/bin/gene However, - Extensions `/bin/generate` must not write to the app directory. - Extensions `` directory is replaced by an `` directory. -- If an extension is missing `/bin/build`, the extension root is treated as a pre-populated `` directory. +- If an extension is missing `/bin/generate`, the extension root is treated as a pre-populated `` directory. After `/bin/generate` executes, the `` directory may contain - `build.toml`, with the same contents as a normal buildpack's `build.toml`, but From fefd3388727c3f36a63a02640e1335ea059e8729 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:39:02 -0400 Subject: [PATCH 118/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 11cf5d3dc..73f253f16 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -166,8 +166,6 @@ RUN groupadd cnb --gid ${CNB_GROUP_ID} && \ useradd --uid ${CNB_USER_ID} --gid ${CNB_GROUP_ID} -m -s /bin/bash cnb USER ${CNB_USER_ID}:${CNB_GROUP_ID} - -COPY genpkgs /cnb/image/genpkgs ``` `run.Dockerfile` for use with the example `app.run.Dockerfile.out` extension that always installs the latest version of curl: From 74767a099dc754369c34a8f744f2583a8ae8905d Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:39:10 -0400 Subject: [PATCH 119/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 73f253f16..bca24fec0 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -207,7 +207,6 @@ RUN curl -L https://example.com/mypkg-install | sh # installs to /opt # Unresolved Questions [unresolved-questions]: #unresolved-questions -- Should `genpkgs` be part of this proposal? Opinion: Yes, otherwise it's difficult to maintain a valid SBoM. - Should we allow base images to provide build plan entries so that extensions aren't required to satisfy buildpacks? Opinion: Not yet, no-op extension can be used for now. # Spec. Changes (OPTIONAL) From 3b73177c87dc6d489f9247612fe03bd3234f1103 Mon Sep 17 00:00:00 2001 From: Stephen Levine Date: Wed, 22 Jun 2022 13:39:18 -0400 Subject: [PATCH 120/372] Update text/0000-dockerfiles.md Co-authored-by: Natalie Arellano Signed-off-by: Stephen Levine --- text/0000-dockerfiles.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index bca24fec0..89a7cdf30 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -213,3 +213,14 @@ RUN curl -L https://example.com/mypkg-install | sh # installs to /opt [spec-changes]: #spec-changes This RFC requires extensive changes to all specifications. + +To deliver incremental value and gather feedback as we implement this large feature, a phased implementation is proposed: +* Phase 1: run.Dockerfiles can be used to switch the runtime base image +* Phase 2: build.Dockerfiles can be used to extend the build time base image + * 2a: `pack` applies the build.Dockerfiles + * 2b: the lifecycle applies the build.Dockerfiles using kaniko +* Phase 3: Dockerfiles and / or run.Dockerfiles can be used to extend the runtime base image + * 3a: `pack` applies the run.Dockerfiles + * 3b: the lifecycle applies the run.Dockerfiles using kaniko + +https://github.com/buildpacks/spec/pull/307 and https://github.com/buildpacks/spec/pull/308 describe the spec changes needed for phase 1. https://github.com/buildpacks/spec/pull/298 approximately describes the spec changes needed for all phases. From 87092f857b171e2a2a2bbb2f1ec79cd16a5f1d9f Mon Sep 17 00:00:00 2001 From: Andrew Gracey Date: Mon, 27 Jun 2022 10:28:02 -0700 Subject: [PATCH 121/372] Update text/0000-additonal-oci-artifacts.md Co-authored-by: Natalie Arellano --- text/0000-additonal-oci-artifacts.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md index 71140e541..be099efa8 100644 --- a/text/0000-additonal-oci-artifacts.md +++ b/text/0000-additonal-oci-artifacts.md @@ -20,7 +20,8 @@ Allow buildpack authors to specify additional artifacts from their buildpacks to Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with. - WebAssembly (WASM) -- Portable binaries to be run with a WebAssembly virtual machine. -- OCI Artifact -- Any blob of data that can be stored in an OCI registry +- [OCI Artifact](https://github.com/opencontainers/artifacts/blob/main/definitions-terms.md#media-type) -- Any blob of data that can be stored in an OCI registry. From the spec: +> An artifact has a type, which has a collection of layers. The Artifact is defined as unique by its `manifest.config.mediaType`. Layers are defined by their `layer.config.mediaType`. # Motivation [motivation]: #motivation From aa04b64de1a128400cd8f51354342b994bd2b3ca Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Mon, 27 Jun 2022 16:41:52 -0400 Subject: [PATCH 122/372] RFC for SBOM describing lifecycle / launcher Signed-off-by: Natalie Arellano --- text/0000-launcher-sbom.md | 146 +++++++++++++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 text/0000-launcher-sbom.md diff --git a/text/0000-launcher-sbom.md b/text/0000-launcher-sbom.md new file mode 100644 index 000000000..ce1671476 --- /dev/null +++ b/text/0000-launcher-sbom.md @@ -0,0 +1,146 @@ +# Meta + +[meta]: #meta + +- Name: SBOM for lifecycle / launcher +- Start Date: 2022-06-27 +- Author(s): natalieparellano +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary + +[summary]: #summary + +Today, SBOMs produced for CNB-built images do not include information describing the CNB lifecycle itself - including +the launcher, a component in the final image. This RFC proposes a mechanism by which the lifecycle can provide an SBOM +describing itself (as a build-time dependency) and the launcher (as a runtime dependency). + +# Definitions + +[definitions]: #definitions + +SBOM: A list of components in a piece of software. Software vendors often create products by assembling open source and +commercial software components. The SBOM describes the components in a product. + +lifecycle: software that orchestrates a CNB build. + +launcher: an executable that is the `ENTRYPOINT` of the exported OCI image. It is used to start processes at runtime. + +# Motivation + +[motivation]: #motivation + +- Why should we do this? A more complete SBOM for CNB-built images. +- What use cases does it support? As an end user, I want SBOM information for every component of my software supply + chain. + +# What it is + +[what-it-is]: #what-it-is + +The SBOM for a CNB-built image could be broken down as follows: + +* **Base image dependencies** + * The platform is responsible for providing SBOM information (see the closure + of https://github.com/buildpacks/rfcs/pull/211 in favor of https://github.com/buildpacks/rfcs/pull/195). Out of + scope for this RFC. +* **Buildpack-provided dependencies** + * SBOM files output by buildpacks are copied to `/sbom/build/` + and `/sbom/run/`. `/sbom/run` is a layer in the final image. + See [RFC 0095](https://github.com/buildpacks/rfcs/blob/main/text/0095-sbom.md). +* **CNB lifecycle** + * Currently there is no SBOM information associated to the image. A CycloneDX SBOM is included in lifecycle + [releases](https://github.com/buildpacks/lifecycle/releases), but it is left up to individual end users to locate + this information. The subject of this RFC. + +## Proposed Changes: + +* The lifecycle will ship with SBOM files in CycloneDX, SPDX, and Syft formats. + * [Lifecycle images](https://hub.docker.com/r/buildpacksio/lifecycle) will include the following SBOM files: + * `/cnb/lifecycle.sbom.cdx.json` + * `/cnb/lifecycle.sbom.spdx.json` + * `/cnb/lifecycle.sbom.syft.json` + * `/cnb/launcher.sbom.cdx.json` + * `/cnb/launcher.sbom.spdx.json` + * `/cnb/launcher.sbom.syft.json` + * Analogously, [lifecycle tarballs](https://github.com/buildpacks/lifecycle/releases) will include these files + rooted at `./lifecycle` - e.g., `/cnb/lifecycle.sbom.cdx.json` above will be `./lifecycle/lifecycle.sbom.cdx.json` + in the tarball. + +* Builder authors should include the new SBOM files in builder images. + * There shouldn't be any changes required to `pack builder create` as `pack` takes a lifecycle tarball as an input, + and the files will be there in the tarball. + +* Before `export`, the lifecycle will copy the above files to: + * `/cnb/lifecycle.sbom.cdx.json` -> `/sbom/build/cnb_lifecycle/sbom.cdx.json` + * `/cnb/lifecycle.sbom.spdx.json` -> `/sbom/build/cnb_lifecycle/sbom.spdx.json` + * `/cnb/lifecycle.sbom.syft.json` -> `/sbom/build/cnb_lifecycle/sbom.syft.json` + * `/cnb/launcher.sbom.cdx.json` -> `/sbom/run/cnb_lifecycle/sbom.cdx.json` + * `/cnb/launcher.sbom.spdx.json` -> `/sbom/run/cnb_lifecycle/sbom.spdx.json` + * `/cnb/launcher.sbom.syft.json` -> `/sbom/run/cnb_lifecycle/sbom.syft.json` + +* `cnb/lifecycle` will become a reserved buildpack ID + +* Because they are a part of `/sbom/run`, SBOM files describing the launcher will be exported in the final + image. + * SBOM files in `/sbom/build` may be saved off by the platform before the build container exits (no changes + to the existing workflow). + +* If no SBOM files are found in `/cnb` (e.g., if the builder author did not include them), the lifecycle will warn and + continue. Alternatively, the lifecycle could generate the files on the fly, but this would increase build times. + * If there is a previous image, `/sbom/run/cnb_lifecycle` may contain SBOM files - but these should + be deleted before `export` as there is no guarantee that the lifecycle that created the previous image is the same + as the current lifecycle. + +* In theory, there should be no changes needed for end-users to consume the new SBOM files, as the files will be placed + in the same directory, with the same structure, as SBOM files for buildpack-provided dependencies. + +# Drawbacks + +[drawbacks]: #drawbacks + +Why should we *not* do this? More work for the lifecycle. + +# Alternatives + +[alternatives]: #alternatives + +- What other designs have been considered? In theory, we don't actually need the files to be + in `/sbom//cnb_lifecycle` - because any files restored from a previous build will be ignored (see + above). They could be annotations or attestations on the image instead ( + see https://github.com/buildpacks/rfcs/pull/195) - but this would require platforms to keep track of the files as + inputs to the signer binary. + +- Why is this proposal the best? It is easy and straightforward for the lifecycle to copy pre-generated files + to `/sbom//cnb_lifecycle`. + +- What is the impact of not doing this? A less complete SBOM for CNB-built images. + +# Prior Art + +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +* RFCs: + * Base image dependencies: https://github.com/buildpacks/rfcs/pull/211, https://github.com/buildpacks/rfcs/pull/195 + * Buildpack-provided dependencies: [RFC 0095](https://github.com/buildpacks/rfcs/blob/main/text/0095-sbom.md) + +# Unresolved Questions + +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved through implementation of the feature? + - Tooling or libraries used by the lifecycle to generate SBOM files describing itself. + +# Spec. Changes (OPTIONAL) + +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. + +The platform spec should make mention of the new SBOM files. The buildpack spec should note that `cnb_lifecycle` is a +reserved buildpack ID. From 5b1d9732aa73cd2f0bf34562db131a0d2ea82127 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 10:35:53 -0400 Subject: [PATCH 123/372] Add a discarded alternative Signed-off-by: Natalie Arellano --- text/0000-launcher-sbom.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/text/0000-launcher-sbom.md b/text/0000-launcher-sbom.md index ce1671476..95327dd7c 100644 --- a/text/0000-launcher-sbom.md +++ b/text/0000-launcher-sbom.md @@ -92,9 +92,9 @@ The SBOM for a CNB-built image could be broken down as follows: * If no SBOM files are found in `/cnb` (e.g., if the builder author did not include them), the lifecycle will warn and continue. Alternatively, the lifecycle could generate the files on the fly, but this would increase build times. - * If there is a previous image, `/sbom/run/cnb_lifecycle` may contain SBOM files - but these should - be deleted before `export` as there is no guarantee that the lifecycle that created the previous image is the same - as the current lifecycle. + * If there is a previous image, `/sbom/run/cnb_lifecycle` may contain SBOM files - but these should be + deleted before `export` as there is no guarantee that the lifecycle that created the previous image is the same as + the current lifecycle. * In theory, there should be no changes needed for end-users to consume the new SBOM files, as the files will be placed in the same directory, with the same structure, as SBOM files for buildpack-provided dependencies. @@ -109,11 +109,16 @@ Why should we *not* do this? More work for the lifecycle. [alternatives]: #alternatives -- What other designs have been considered? In theory, we don't actually need the files to be - in `/sbom//cnb_lifecycle` - because any files restored from a previous build will be ignored (see - above). They could be annotations or attestations on the image instead ( - see https://github.com/buildpacks/rfcs/pull/195) - but this would require platforms to keep track of the files as - inputs to the signer binary. +- What other designs have been considered? + - This could all be implemented with a buildpack - e.g., a `cnb/lifecycle` utility buildpack whose sole + responsibility would be to copy the SBOM files from `/cnb/lifecycle` to `/sbom`. However, this would + introduce quite a bit of complexity and overhead for what is ultimately a very simple operation. A potential + benefit is that platforms using Platform API `0.8` and higher wouldn't need to upgrade - but they would need to + add the new utility buildpack to builders, which is probably just as much work. + - In theory, we don't actually need the files to be in `/sbom//cnb_lifecycle` - because any files + restored from a previous build will be ignored (see above). They could be annotations or attestations on the image + instead (see https://github.com/buildpacks/rfcs/pull/195) - but this would require platforms to keep track of the + files as inputs to the signer binary. - Why is this proposal the best? It is easy and straightforward for the lifecycle to copy pre-generated files to `/sbom//cnb_lifecycle`. From 400d8854a3fdf6ac9e4db6e27e676470fbdbfe3c Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 13:06:51 -0400 Subject: [PATCH 124/372] First pass Signed-off-by: Natalie Arellano --- text/0000-deprecate-apis.md | 134 ++++++++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+) create mode 100644 text/0000-deprecate-apis.md diff --git a/text/0000-deprecate-apis.md b/text/0000-deprecate-apis.md new file mode 100644 index 000000000..6bd7d7896 --- /dev/null +++ b/text/0000-deprecate-apis.md @@ -0,0 +1,134 @@ +# Meta + +[meta]: #meta + +- Name: Deprecate old buildpack and platform APIs +- Start Date: 2022-06-28 +- Author(s): natalieparellano +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary + +[summary]: #summary + +Today, the CNB project defines XXX buildpack APIs and XXX platform APIs. The CNB lifecycle, for backwards compatibility +and in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all +XXX APIs and all XXX combinations of buildpack and platform APIs. This is a maintenance burden. + +Additionally, as we've talked about adding new features to the project, it has become difficult to determine how these +additions might be compatible with older APIs. This has slowed down progress within the project. + +This RFC proposes: + +* Marking as deprecated the following APIs: + * buildpack APIs XXX and older + * platform APIs XXX and older +* A general policy of: + * Removing support for deprecated APIs 6 months after those APIs are marked as deprecated + +# Definitions + +[definitions]: #definitions + +Buildpack API: XXX + +Platform API: XXX + +lifecycle: XXX + +# Motivation + +[motivation]: #motivation + +- Why should we do this? Lower maintenance burden, enable faster development of new APIs. +- What use cases does it support? As a lifecycle maintainer, I only want to support the latest buildpack and platform + APIs. As a CNB contributor, when thinking about new features, I want a smaller set of supported APIs to care about. +- What is the expected outcome? Hopefully, not too much inconvenience for buildpack authors and platform operators, + because: + - Hopefully they are already on newer versions of the APIs, but if not... + - We will have socialized this change appropriately (e.g., in Slack, through the mailing list, GitHub, etc.) + - The `CNB_DEPRECATION_MODE` setting will have alerted them that upgrading is necessary + - We will have published migration guides to help them upgrade + - 6 months is a reasonable amount of time to complete this process + +# What it is + +[what-it-is]: #what-it-is + +APIs will be marked as deprecated here: + +``` +XXX +``` + +As described in XXX: + +* If `CNB_DEPRECATION_MODE` is set, XXX. +* If `CNB_DEPRECATION_MODE` is unset, XXX. + +After 6 months, the values will be: + +``` +XXX +``` + +* If a buildpack tries to use an unsupported API, the lifecycle will fail with the following message: XXX. +* If a platform tries to use an unsupported API, the lifecycle will fail with the following message: XXX. + +# Migration + +[migration]: #migration + +### Buildpack API + +Buildpack authors will need to update buildpacks using the old APIs to a newer API. These buildpacks will need to be +re-published and re-discovered. + +Platform authors / builder authors will need to re-create builders using the newer buildpacks. + +End-users will need to consumer the newer buildpacks and / or builders. + +### Platform API + +Platform authors will need to update their platform implementations to use the newer platform API. The appropriate value +of `CNB_PLATFORM_API` must be set in the lifecycle's execution environment. + +End-users will need to update their usage of the newer images if they were formerly using a platform API < 0.XXX (see +XXX). + +# Drawbacks + +[drawbacks]: #drawbacks + +Why should we *not* do this? + +* This will inevitably cause extra work for a fraction of buildpacks users. + +# Alternatives + +[alternatives]: #alternatives + +- What other designs have been considered? Doing nothing. +- Why is this proposal the best? If we want to make progress toward a 1.0 version of the spec, we need to start dropping + older APIs. +- What is the impact of not doing this? Slower progress in the project due to the maintenance burden and difficulty + conceptualizing compatibility concerns for so many APIs. + +# Prior Art + +[prior-art]: #prior-art + +golang support schedule: XXX + +RFC that was put "on pause" due to difficulties managing API complexities: XXX + +# Spec. Changes (OPTIONAL) + +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. + +We should clearly mark the deprecated specs as deprecated on their respective branches. From 8e2dacf87aec2c02b92736cde98e3dd1e1aac215 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 17:16:49 -0400 Subject: [PATCH 125/372] Added some more detail Signed-off-by: Natalie Arellano --- text/0000-deprecate-apis.md | 100 +++++++++++++++++++++++++++--------- 1 file changed, 77 insertions(+), 23 deletions(-) diff --git a/text/0000-deprecate-apis.md b/text/0000-deprecate-apis.md index 6bd7d7896..500011369 100644 --- a/text/0000-deprecate-apis.md +++ b/text/0000-deprecate-apis.md @@ -15,18 +15,20 @@ [summary]: #summary -Today, the CNB project defines XXX buildpack APIs and XXX platform APIs. The CNB lifecycle, for backwards compatibility -and in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all -XXX APIs and all XXX combinations of buildpack and platform APIs. This is a maintenance burden. +Today, the CNB project defines 7 buildpack APIs and 7 platform APIs. The CNB lifecycle, for backwards compatibility and +in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all 14 +APIs and all 49 combinations of buildpack and platform APIs. This is a maintenance burden. Additionally, as we've talked about adding new features to the project, it has become difficult to determine how these additions might be compatible with older APIs. This has slowed down progress within the project. +As we progress toward 1.0, it seems prudent to start deprecating APIs from which we have made breaking changes. + This RFC proposes: * Marking as deprecated the following APIs: - * buildpack APIs XXX and older - * platform APIs XXX and older + * buildpack APIs 0.6 and older + * platform APIs 0.6 and older * A general policy of: * Removing support for deprecated APIs 6 months after those APIs are marked as deprecated @@ -34,22 +36,45 @@ This RFC proposes: [definitions]: #definitions -Buildpack API: XXX +**lifecycle**: software that orchestrates a CNB build; it executes in a series of phases that each have a distinct +responsibility + +The CNB project distributes the lifecycle in two ways: + +* As a **lifecycle image**: a minimal [image](https://hub.docker.com/r/buildpacksio/lifecycle) containing the lifecycle + binaries +* As a **lifecycle tarball**: a [`.tgz` file](https://github.com/buildpacks/lifecycle/releases) containing the lifecycle + binaries + +The [**lifecycle +descriptor**](https://github.com/buildpacks/rfcs/blob/main/text/0049-multi-api-lifecycle-descriptor.md#lifecycle-descriptor) +is data describing the lifecycle and the APIs that it supports. It is contained in the `io.buildpacks.lifecycle.apis` +and `io.buildpacks.lifecycle.version` labels on lifecycle images, and the `lifecycle.toml` file in lifecycle tarballs. -Platform API: XXX +**platform**: system or software that orchestrates the lifecycle by invoking each lifecycle phase in order -lifecycle: XXX +**Buildpack API**: specifies the interface between a lifecycle program and one or more buildpacks + +**Platform API**: specifies the interface between a lifecycle program and a platform # Motivation [motivation]: #motivation - Why should we do this? Lower maintenance burden, enable faster development of new APIs. + - We should draw the boundary at the 0.6 APIs because: + - They are both over a year old (March 2021) + - For buildpacks, the 0.6 API is the last API with unstandardized SBOMs. + - For platforms, the 0.6 API is the last API where `detect` happens before `analyze`. + - What use cases does it support? As a lifecycle maintainer, I only want to support the latest buildpack and platform APIs. As a CNB contributor, when thinking about new features, I want a smaller set of supported APIs to care about. + - What is the expected outcome? Hopefully, not too much inconvenience for buildpack authors and platform operators, because: - - Hopefully they are already on newer versions of the APIs, but if not... + - Hopefully they are already on newer versions of the APIs ( + our [user survey](https://docs.google.com/presentation/d/10CBBld2VV0iCfrYPMbFI4--kh9UZ9mJVXhE1p1Cjqls/edit#slide=id.p) + supports this), but if not... - We will have socialized this change appropriately (e.g., in Slack, through the mailing list, GitHub, etc.) - The `CNB_DEPRECATION_MODE` setting will have alerted them that upgrading is necessary - We will have published migration guides to help them upgrade @@ -59,25 +84,50 @@ lifecycle: XXX [what-it-is]: #what-it-is -APIs will be marked as deprecated here: +`io.buildpacks.lifecycle.apis` would contain the following: ``` -XXX +{ + "buildpack": { + "deprecated": [ "0.2", "0.3", "0.4", "0.5", "0.6" ], + "supported": [ "0.2", "0.3", "0.4", "0.5", "0.6", "0.7", "0.8" ] + }, + "platform": { + "deprecated": [ "0.3", "0.4", "0.5", "0.6" ], + "supported": [ "0.3", "0.4", "0.5", "0.6", "0.7", "0.8", "0.9" ] + } +} ``` -As described in XXX: +As described +in [RFC 0049](https://github.com/buildpacks/rfcs/blob/main/text/0049-multi-api-lifecycle-descriptor.md#lifecycle-descriptor) +, if a buildpack or platform tries to use a deprecated API: -* If `CNB_DEPRECATION_MODE` is set, XXX. -* If `CNB_DEPRECATION_MODE` is unset, XXX. +* If `CNB_DEPRECATION_MODE` is unset, the lifecycle will print a warning and continue +* If `CNB_DEPRECATION_MODE`=`warn`, the lifecycle will print a warning and continue +* If `CNB_DEPRECATION_MODE`=`error`, the lifecycle will fail +* If `CNB_DEPRECATION_MODE`=`silent`, the lifecycle will continue w/o warning -After 6 months, the values will be: +After 6 months, the deprecated APIs would become unsupported and `io.buildpacks.lifecycle.apis` would contain the +following: ``` -XXX +{ + "buildpack": { + "deprecated": [ ], + "supported": [ "0.7", "0.8", "" ] + }, + "platform": { + "deprecated": [ ], + "supported": [ "0.7", "0.8", "0.9", "" ] + } +} ``` -* If a buildpack tries to use an unsupported API, the lifecycle will fail with the following message: XXX. -* If a platform tries to use an unsupported API, the lifecycle will fail with the following message: XXX. +* If a buildpack tries to use an unsupported API, the lifecycle will fail with a message such + as: `buildpack API version '0.6' is incompatible with the lifecycle`. +* If a platform tries to use an unsupported API, the lifecycle will fail with a message such + as: `platform API version '0.6' is incompatible with the lifecycle`. # Migration @@ -97,8 +147,9 @@ End-users will need to consumer the newer buildpacks and / or builders. Platform authors will need to update their platform implementations to use the newer platform API. The appropriate value of `CNB_PLATFORM_API` must be set in the lifecycle's execution environment. -End-users will need to update their usage of the newer images if they were formerly using a platform API < 0.XXX (see -XXX). +End-users will need to update their usage of the newer images if they were formerly using a platform +API < [0.4](https://github.com/buildpacks/spec/releases/tag/platform%2Fv0.4) (see +[RFC 0045](https://github.com/buildpacks/rfcs/blob/main/text/0045-launcher-arguments.md)). # Drawbacks @@ -122,9 +173,12 @@ Why should we *not* do this? [prior-art]: #prior-art -golang support schedule: XXX - -RFC that was put "on pause" due to difficulties managing API complexities: XXX +* [Ubuntu release cycle](https://ubuntu.com/about/release-cycle) +* [Golang release policy](https://go.dev/doc/devel/release#policy) +* [Docker Community Edition release cadence](https://www.serverwatch.com/server-news/docker-18-06-ce-debuts-alongside-new-release-cadence/) +* [Kubernetes support window](https://kubernetes.io/blog/2020/08/31/kubernetes-1-19-feature-one-year-support/) +* CNB RFC that was put "on pause" due to difficulties managing API + complexities: https://github.com/buildpacks/rfcs/pull/145 # Spec. Changes (OPTIONAL) From b720b0c70fbd664dbf81b9ed214f60b10ff10fbd Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 17:23:40 -0400 Subject: [PATCH 126/372] Added some links Signed-off-by: Natalie Arellano --- text/0000-deprecate-apis.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/text/0000-deprecate-apis.md b/text/0000-deprecate-apis.md index 500011369..9aabbe3f3 100644 --- a/text/0000-deprecate-apis.md +++ b/text/0000-deprecate-apis.md @@ -15,9 +15,10 @@ [summary]: #summary -Today, the CNB project defines 7 buildpack APIs and 7 platform APIs. The CNB lifecycle, for backwards compatibility and -in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all 14 -APIs and all 49 combinations of buildpack and platform APIs. This is a maintenance burden. +Today, the CNB project [defines](https://github.com/buildpacks/spec/releases) 7 buildpack APIs and 7 platform APIs. The +CNB lifecycle, for backwards compatibility and in order to enable buildpacks and platforms to upgrade their respective +APIs independently, currently supports all 14 APIs and all 49 combinations of buildpack and platform APIs. This is a +maintenance burden. Additionally, as we've talked about adding new features to the project, it has become difficult to determine how these additions might be compatible with older APIs. This has slowed down progress within the project. @@ -36,7 +37,7 @@ This RFC proposes: [definitions]: #definitions -**lifecycle**: software that orchestrates a CNB build; it executes in a series of phases that each have a distinct +[**lifecycle**](https://github.com/buildpacks/lifecycle): software that orchestrates a CNB build; it executes in a series of phases that each have a distinct responsibility The CNB project distributes the lifecycle in two ways: @@ -53,9 +54,9 @@ and `io.buildpacks.lifecycle.version` labels on lifecycle images, and the `lifec **platform**: system or software that orchestrates the lifecycle by invoking each lifecycle phase in order -**Buildpack API**: specifies the interface between a lifecycle program and one or more buildpacks +[**Buildpack API**](https://github.com/buildpacks/spec/blob/main/buildpack.md): specifies the interface between a lifecycle program and one or more buildpacks -**Platform API**: specifies the interface between a lifecycle program and a platform +[**Platform API**](https://github.com/buildpacks/spec/blob/main/platform.md): specifies the interface between a lifecycle program and a platform # Motivation @@ -77,7 +78,7 @@ and `io.buildpacks.lifecycle.version` labels on lifecycle images, and the `lifec supports this), but if not... - We will have socialized this change appropriately (e.g., in Slack, through the mailing list, GitHub, etc.) - The `CNB_DEPRECATION_MODE` setting will have alerted them that upgrading is necessary - - We will have published migration guides to help them upgrade + - We will have published [migration guides](https://buildpacks.io/docs/reference/spec/migration/) to help them upgrade - 6 months is a reasonable amount of time to complete this process # What it is From 4abb253d219eb6cc1dd32d06e4af3f3a3f801760 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 5 Jul 2022 15:46:24 -0400 Subject: [PATCH 127/372] Add a note about resolving registry credentials up front Signed-off-by: Natalie Arellano --- text/0000-dockerfiles.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 89a7cdf30..4539e1b81 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -40,6 +40,8 @@ Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to ap * build * export +When Dockerfiles are used to update the run image, care should be taken to ensure registry access prior to the `build` phase, to avoid long builds that fail due to incorrect credentials. + ### Dynamically-applied Dockerfiles A builder image may include any number of "extensions" directories in `/cnb/extensions/`. From f8e7bfa555c302ac9203e8ccfa0ab29e7987e5c3 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 6 Jul 2022 11:24:44 -0400 Subject: [PATCH 128/372] Updates per Emily's feedback Signed-off-by: Natalie Arellano --- text/0000-dockerfiles.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 4539e1b81..6f5afb977 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -32,10 +32,9 @@ For a given application, a build that uses extensions could be optimized by crea Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. The order of operations would be something like the following: * analyze -* detect -* run extensions' bin/generate, output Dockerfiles are written to a volume -* apply Dockerfiles to run image (could run in parallel with build image extension) -* apply Dockerfiles to build image +* detect - after standard detection detect will also run extensions' bin/generate, output Dockerfiles are written to a volume +* apply Dockerfiles to run image (could run in parallel with builder image extension) +* apply Dockerfiles to builder image * restore * build * export @@ -88,9 +87,10 @@ However, - If an extension is missing `/bin/generate`, the extension root is treated as a pre-populated `` directory. After `/bin/generate` executes, the `` directory may contain -- `build.toml`, with the same contents as a normal buildpack's `build.toml`, but +- `build.toml`, with the same contents as a normal buildpack's `build.toml` (the `unmet` table array), but - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` or `Dockerfile` -- `launch.toml`, with the same contents as a normal buildpack's `launch.toml`, but +- `launch.toml`, + - Without the `labels` table array - Without the `processes` table array - Without the `slices` table array - With an additional `args` table array with `name` and `value` fields that are provided as build args to `run.Dockerfile` or `Dockerfile` @@ -101,7 +101,7 @@ Support for other instruction formats, e.g., LLB JSON files, could be added in t `build.Dockerfile`, `run.Dockerfile`, and `Dockerfile` target the builder image, runtime base image, or both base images, respectively. -If no Dockerfiles are present, `/bin/generate` may still consume build plan entries and add metadata to `build.toml`/`launch.toml`. +If no Dockerfiles are present, `/bin/generate` may still consume build plan entries. Dockerfiles are applied to their corresponding base images after all extensions are executed and before any regular buildpacks are executed. Dockerfiles are applied in the order determined during buildpack detection. When multiple Dockerfiles are applied, the intermediate image generated from the application of the current Dockerfile will be provided as the `base_image` ARG to the next Dockerfile. Dockerfiles that target the run image (only) may ignore the provided `base_image` (e.g., `FROM some-other-image`). Dockerfiles that change the runtime base image may still use `COPY --from=${base_image}`. From 3a26570c2c4a2634b14832282828178a38782493 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 7 Jul 2022 11:23:03 -0400 Subject: [PATCH 129/372] Updates per 7/7/22 Working Group - Change launch.toml -> run.toml since there are no overlapping fields between buildpacks and extensions - Remove Dockerfile without a prefix since the args must be specified in either build.toml or run.toml - Change must -> should for defaulting build_id - Nest pre-populated output files under detect or generate (future proofing) - Mention how we'll preserve top layer information for rebase Signed-off-by: Natalie Arellano --- text/0000-dockerfiles.md | 52 +++++++++++++++++++--------------------- 1 file changed, 24 insertions(+), 28 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 6f5afb977..e6cdcbc28 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -17,7 +17,7 @@ This RFC introduces functionality for customizing base images, as an alternative # Motivation [motivation]: #motivation -[RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md) introduces complexity by defining an API that allows buildpacks to modify base images. To avoid this complexity, we could rely on generated Dockerfiles for base image manipulation. This would simplify the original proposal by, e.g., only requiring a copy of the buildpack on the build-time base image. +[RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md) introduces complexity by defining an API that allows buildpacks to modify base images. To avoid this complexity, we could rely on generated Dockerfiles for base image manipulation. This would simplify the original proposal by, e.g., only requiring a copy of the extension on the build-time base image. # What it is [what-it-is]: #what-it-is @@ -31,13 +31,13 @@ For a given application, a build that uses extensions could be optimized by crea [how-it-works]: #how-it-works Note: kaniko, buildah, BuildKit, or the original Docker daemon may be used to apply Dockerfiles at the platform's discretion. The order of operations would be something like the following: -* analyze -* detect - after standard detection detect will also run extensions' bin/generate, output Dockerfiles are written to a volume -* apply Dockerfiles to run image (could run in parallel with builder image extension) -* apply Dockerfiles to builder image -* restore -* build -* export +* `analyze` +* `detect` - after standard detection detect will also run extensions' bin/generate, output Dockerfiles are written to a volume +* `extend` - applies run.Dockerfiles to run image (could run in parallel with builder image extension) +* `extend` - applies build.Dockerfiles to builder image +* `restore` +* `build` +* `export` When Dockerfiles are used to update the run image, care should be taken to ensure registry access prior to the `build` phase, to avoid long builds that fail due to incorrect credentials. @@ -74,7 +74,7 @@ Unlike buildpacks, Extensions participate in the buildpack detection process, with the same UID, GID, and interface for `/bin/detect`. However, - `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. -- If an extension is missing `/bin/detect`, the extension root is treated as a pre-populated output directory (i.e., extensions can include a static build plan). +- If an extension is missing `/bin/detect`, the extension root `./detect` directory is treated as a pre-populated output directory (i.e., extensions can include a static build plan). - Extensions may only output `provides` entries to the build plan. They must not output `requires`. - Extensions are not included in `order` definitions (e.g., in `builder.toml`); instead, a separate `order-extensions` table should be used. The `order-extensions` table will be prepended to each group in the provided `order` (as if `order-extensions` were a composite buildpack). - Extensions are always `optional`. @@ -84,22 +84,18 @@ To generate these Dockerfiles, the lifecycle executes the extension's `/bin/gene However, - Extensions `/bin/generate` must not write to the app directory. - Extensions `` directory is replaced by an `` directory. -- If an extension is missing `/bin/generate`, the extension root is treated as a pre-populated `` directory. +- If an extension is missing `/bin/generate`, the extension root `./generate` directory is treated as a pre-populated `` directory. After `/bin/generate` executes, the `` directory may contain - `build.toml`, with the same contents as a normal buildpack's `build.toml` (the `unmet` table array), but - - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` or `Dockerfile` -- `launch.toml`, - - Without the `labels` table array - - Without the `processes` table array - - Without the `slices` table array - - With an additional `args` table array with `name` and `value` fields that are provided as build args to `run.Dockerfile` or `Dockerfile` - -- Either `Dockerfile` or either or both of `build.Dockerfile` and `run.Dockerfile` + - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` +- `run.toml`, + - With an `args` table array with `name` and `value` fields that are provided as build args to `run.Dockerfile` +- Either or both of `build.Dockerfile` and `run.Dockerfile` Support for other instruction formats, e.g., LLB JSON files, could be added in the future. -`build.Dockerfile`, `run.Dockerfile`, and `Dockerfile` target the builder image, runtime base image, or both base images, respectively. +`build.Dockerfile` and `run.Dockerfile`target the builder image or runtime base image, respectively. If no Dockerfiles are present, `/bin/generate` may still consume build plan entries. @@ -108,13 +104,13 @@ Dockerfiles are applied in the order determined during buildpack detection. When All Dockerfiles are provided with `base_image` and `build_id` args. The `base_image` arg allows the Dockerfile to reference the original base image. -The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and must be defaulted to `0`. The executor of the Dockerfile will provide the `build_id` as a UUID (this eliminates the need to track this variable). +The `build_id` arg allows the Dockerfile to invalidate the cache after a certain layer and should be defaulted to `0`. The executor of the Dockerfile will provide the `build_id` as a UUID (this eliminates the need to track this variable). When the `$build_id` arg is referenced in a `RUN` instruction, all subsequent layers will be rebuilt on the next build (as the value will change). -Build args specified in `build.toml` are provided to `build.Dockerfile` or `Dockerfile` (when applied to the build-time base image). -Build args specified in `launch.toml` are provided to `run.Dockerfile` or `Dockerfile` (when applied to the runtime base image). +Build args specified in `build.toml` are provided to `build.Dockerfile` (when applied to the build-time base image). +Build args specified in `run.toml` are provided to `run.Dockerfile` (when applied to the runtime base image). -A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically on the base image before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. If multiple Dockerfiles are applied, all must set `io.buildpacks.rebasable=true` for the final value to be `true`. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. +A runtime base image may indicate that it preserves ABI compatibility by adding the label `io.buildpacks.rebasable=true`. In the case of builder-specified Dockerfiles, `io.buildpacks.rebasable=false` is set automatically on the base image before a runtime Dockerfile is applied and must be explicitly set to `true` if desired. If multiple Dockerfiles are applied, all must set `io.buildpacks.rebasable=true` for the final value to be `true`. Rebasing an app without this label set to `true` requires passing a new `--force` flag to `pack rebase`. When the run image is extended and `io.buildpacks.rebasable=true`, the `extend` phase will communicate to the `export` phase the top layer of the run image (prior to extension) so that the exporter can set the appropriate value of `io.buildpacks.lifecycle.metadata` `runImage.topLayer`. #### Example: App-specified Dockerfile Extension @@ -137,8 +133,8 @@ Note: The Dockerfiles referenced must disable rebasing, and build times will be ##### `/cnb/ext/com.example.rpmext/bin/generate` ``` #!/bin/sh -[ -f Gemfile.lock ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-ruby" "$1/Dockerfile" -[ -f package.json ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-node" "$1/Dockerfile" +[ -f Gemfile.lock ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-ruby" "$1/build.Dockerfile" +[ -f package.json ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-node" "$1/build.Dockerfile" ``` @@ -170,7 +166,7 @@ RUN groupadd cnb --gid ${CNB_GROUP_ID} && \ USER ${CNB_USER_ID}:${CNB_GROUP_ID} ``` -`run.Dockerfile` for use with the example `app.run.Dockerfile.out` extension that always installs the latest version of curl: +`run.Dockerfile` that always installs the latest version of curl: ``` ARG base_image FROM ${base_image} @@ -182,7 +178,7 @@ RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/* ``` (note: this Dockerfile disables rebasing, as OS package installation is not rebasable) -`run.Dockerfile` for use with the example `app.run.Dockerfile.out` extension that installs a special package to /opt: +`run.Dockerfile` that installs a special package to /opt: ``` ARG base_image FROM ${base_image} @@ -221,7 +217,7 @@ To deliver incremental value and gather feedback as we implement this large feat * Phase 2: build.Dockerfiles can be used to extend the build time base image * 2a: `pack` applies the build.Dockerfiles * 2b: the lifecycle applies the build.Dockerfiles using kaniko -* Phase 3: Dockerfiles and / or run.Dockerfiles can be used to extend the runtime base image +* Phase 3: run.Dockerfiles can be used to extend the runtime base image * 3a: `pack` applies the run.Dockerfiles * 3b: the lifecycle applies the run.Dockerfiles using kaniko From c2cac2e009c4fcfb935901b0ae58b2516e5c4bbc Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 7 Jul 2022 16:37:18 -0400 Subject: [PATCH 130/372] Updates per Emily's feedback Signed-off-by: Natalie Arellano --- text/0000-launcher-sbom.md | 59 +++++++++++++++++++++----------------- 1 file changed, 32 insertions(+), 27 deletions(-) diff --git a/text/0000-launcher-sbom.md b/text/0000-launcher-sbom.md index 95327dd7c..a299dea92 100644 --- a/text/0000-launcher-sbom.md +++ b/text/0000-launcher-sbom.md @@ -61,40 +61,45 @@ The SBOM for a CNB-built image could be broken down as follows: * The lifecycle will ship with SBOM files in CycloneDX, SPDX, and Syft formats. * [Lifecycle images](https://hub.docker.com/r/buildpacksio/lifecycle) will include the following SBOM files: - * `/cnb/lifecycle.sbom.cdx.json` - * `/cnb/lifecycle.sbom.spdx.json` - * `/cnb/lifecycle.sbom.syft.json` - * `/cnb/launcher.sbom.cdx.json` - * `/cnb/launcher.sbom.spdx.json` - * `/cnb/launcher.sbom.syft.json` + * `/cnb/lifecycle/lifecycle.sbom.cdx.json` + * `/cnb/lifecycle/lifecycle.sbom.spdx.json` + * `/cnb/lifecycle/lifecycle.sbom.syft.json` + * `/cnb/lifecycle/launcher.sbom.cdx.json` + * `/cnb/lifecycle/launcher.sbom.spdx.json` + * `/cnb/lifecycle/launcher.sbom.syft.json` * Analogously, [lifecycle tarballs](https://github.com/buildpacks/lifecycle/releases) will include these files - rooted at `./lifecycle` - e.g., `/cnb/lifecycle.sbom.cdx.json` above will be `./lifecycle/lifecycle.sbom.cdx.json` - in the tarball. + rooted at `./lifecycle` - e.g., `/cnb/lifecycle/lifecycle.sbom.cdx.json` above will + be `./lifecycle/lifecycle.sbom.cdx.json` in the tarball. * Builder authors should include the new SBOM files in builder images. * There shouldn't be any changes required to `pack builder create` as `pack` takes a lifecycle tarball as an input, and the files will be there in the tarball. * Before `export`, the lifecycle will copy the above files to: - * `/cnb/lifecycle.sbom.cdx.json` -> `/sbom/build/cnb_lifecycle/sbom.cdx.json` - * `/cnb/lifecycle.sbom.spdx.json` -> `/sbom/build/cnb_lifecycle/sbom.spdx.json` - * `/cnb/lifecycle.sbom.syft.json` -> `/sbom/build/cnb_lifecycle/sbom.syft.json` - * `/cnb/launcher.sbom.cdx.json` -> `/sbom/run/cnb_lifecycle/sbom.cdx.json` - * `/cnb/launcher.sbom.spdx.json` -> `/sbom/run/cnb_lifecycle/sbom.spdx.json` - * `/cnb/launcher.sbom.syft.json` -> `/sbom/run/cnb_lifecycle/sbom.syft.json` + * `/cnb/lifecycle/lifecycle.sbom.cdx.json` -> `/sbom/build/buildpacksio_lifecycle/sbom.cdx.json` + * `/cnb/lifecycle/lifecycle.sbom.spdx.json` -> `/sbom/build/buildpacksio_lifecycle/sbom.spdx.json` + * `/cnb/lifecycle/lifecycle.sbom.syft.json` -> `/sbom/build/buildpacksio_lifecycle/sbom.syft.json` + * `/cnb/lifecycle/launcher.sbom.cdx.json` -> `/sbom/launch/buildpacksio_lifecycle/launcher/sbom.cdx.json` + * `/cnb/lifecycle/launcher.sbom.spdx.json` -> `/sbom/launch/buildpacksio_lifecycle/launcher/sbom.spdx.json` + * `/cnb/lifecycle/launcher.sbom.syft.json` -> `/sbom/launch/buildpacksio_lifecycle/launcher/sbom.syft.json` -* `cnb/lifecycle` will become a reserved buildpack ID +* `buildpacksio/lifecycle` will become a reserved buildpack ID -* Because they are a part of `/sbom/run`, SBOM files describing the launcher will be exported in the final +* The `exporter` when adding the launcher layer should use the layer ID `buildpacksio/lifecycle:launcher`, instead + of `launcher` (what it is today). This provides a clear mapping between layer IDs and SBOM paths. + * For consistency, we may also wish to update the IDs for the `launch.sbom`, `config`, and `process-types` layers to + have the prefix `buildpacksio/lifecycle:`, even though these layers have no associated SBOM. + +* Because they are a part of `/sbom/launch`, SBOM files describing the launcher will be exported in the final image. * SBOM files in `/sbom/build` may be saved off by the platform before the build container exits (no changes to the existing workflow). * If no SBOM files are found in `/cnb` (e.g., if the builder author did not include them), the lifecycle will warn and continue. Alternatively, the lifecycle could generate the files on the fly, but this would increase build times. - * If there is a previous image, `/sbom/run/cnb_lifecycle` may contain SBOM files - but these should be - deleted before `export` as there is no guarantee that the lifecycle that created the previous image is the same as - the current lifecycle. + * If there is a previous image, `/sbom/launch/buildpacksio_lifecycle` may contain SBOM files - but these + should be deleted before `export` as there is no guarantee that the lifecycle that created the previous image is + the same as the current lifecycle. * In theory, there should be no changes needed for end-users to consume the new SBOM files, as the files will be placed in the same directory, with the same structure, as SBOM files for buildpack-provided dependencies. @@ -110,18 +115,18 @@ Why should we *not* do this? More work for the lifecycle. [alternatives]: #alternatives - What other designs have been considered? - - This could all be implemented with a buildpack - e.g., a `cnb/lifecycle` utility buildpack whose sole + - This could all be implemented with a buildpack - e.g., a `buildpacksio/lifecycle` utility buildpack whose sole responsibility would be to copy the SBOM files from `/cnb/lifecycle` to `/sbom`. However, this would introduce quite a bit of complexity and overhead for what is ultimately a very simple operation. A potential benefit is that platforms using Platform API `0.8` and higher wouldn't need to upgrade - but they would need to add the new utility buildpack to builders, which is probably just as much work. - - In theory, we don't actually need the files to be in `/sbom//cnb_lifecycle` - because any files - restored from a previous build will be ignored (see above). They could be annotations or attestations on the image - instead (see https://github.com/buildpacks/rfcs/pull/195) - but this would require platforms to keep track of the - files as inputs to the signer binary. + - In theory, we don't actually need the files to be in `/sbom//buildpacksio_lifecycle` - + because any files restored from a previous build will be ignored (see above). They could be annotations or + attestations on the image instead (see https://github.com/buildpacks/rfcs/pull/195) - but this would require + platforms to keep track of the files as inputs to the signer binary. - Why is this proposal the best? It is easy and straightforward for the lifecycle to copy pre-generated files - to `/sbom//cnb_lifecycle`. + to `/sbom//buildpacksio_lifecycle`. - What is the impact of not doing this? A less complete SBOM for CNB-built images. @@ -147,5 +152,5 @@ Discuss prior art, both the good and bad. [spec-changes]: #spec-changes Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. -The platform spec should make mention of the new SBOM files. The buildpack spec should note that `cnb_lifecycle` is a -reserved buildpack ID. +The platform spec should make mention of the new SBOM files. The buildpack spec should note +that `buildpacksio_lifecycle` is a reserved buildpack ID. From c91480e92500441e1fd16650d1fe1778a155dcb6 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 7 Jul 2022 16:42:50 -0400 Subject: [PATCH 131/372] Mention new flag for providing launcher sbom if own launcher is provided Signed-off-by: Natalie Arellano --- text/0000-launcher-sbom.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/0000-launcher-sbom.md b/text/0000-launcher-sbom.md index a299dea92..57f177934 100644 --- a/text/0000-launcher-sbom.md +++ b/text/0000-launcher-sbom.md @@ -154,3 +154,7 @@ Does this RFC entail any proposed changes to the core specifications or extensio The platform spec should make mention of the new SBOM files. The buildpack spec should note that `buildpacksio_lifecycle` is a reserved buildpack ID. + +The `exporter` and the `creator` will accept a new flag, `-launcher-sbom`, for a directory containing SBOM files for a +provided `-launcher`. The lifecycle will warn if `-launcher` is provided without `-launcher-sbom`. The lifecycle will +ignore `-launcher-sbom` if no `-launcher` is provided. From 841f5fef72773047ad2a24b0731df4155037d26d Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 7 Jul 2022 16:45:30 -0400 Subject: [PATCH 132/372] Update path Signed-off-by: Natalie Arellano --- text/0000-launcher-sbom.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-launcher-sbom.md b/text/0000-launcher-sbom.md index 57f177934..42a15ff3b 100644 --- a/text/0000-launcher-sbom.md +++ b/text/0000-launcher-sbom.md @@ -95,7 +95,7 @@ The SBOM for a CNB-built image could be broken down as follows: * SBOM files in `/sbom/build` may be saved off by the platform before the build container exits (no changes to the existing workflow). -* If no SBOM files are found in `/cnb` (e.g., if the builder author did not include them), the lifecycle will warn and +* If no SBOM files are found in `/cnb/lifecycle` (e.g., if the builder author did not include them), the lifecycle will warn and continue. Alternatively, the lifecycle could generate the files on the fly, but this would increase build times. * If there is a previous image, `/sbom/launch/buildpacksio_lifecycle` may contain SBOM files - but these should be deleted before `export` as there is no guarantee that the lifecycle that created the previous image is From 73de1832092fff1dff0d827f9323744cb3140223 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 13 Jul 2022 15:37:20 -0400 Subject: [PATCH 133/372] Apply suggestions from code review Co-authored-by: Emily Casey Signed-off-by: Natalie Arellano --- text/0000-dockerfiles.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index e6cdcbc28..23e9eebd7 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -45,7 +45,7 @@ When Dockerfiles are used to update the run image, care should be taken to ensur A builder image may include any number of "extensions" directories in `/cnb/extensions/`. -Extensions are similar to buildpacks: they have two executables: `/bin/detect` and `/bin/generate`. The interface for these executables is similar to a buildpack's `/bin/detect` and `/bin/build`. +Extensions are similar to buildpacks: they have two executables: `./bin/detect` and `./bin/generate`. The interface for these executables is similar to a buildpack's `./bin/detect` and `./bin/build`. However, instead of a `buildpack.toml` file, extensions have a `extension.toml` file: ```toml api = "" @@ -71,22 +71,22 @@ Unlike buildpacks, - Extensions must not be included in a meta-buildpacks - Extensions must not have `order`/`group` definitions in `extension.toml` -Extensions participate in the buildpack detection process, with the same UID, GID, and interface for `/bin/detect`. +Extensions participate in the buildpack detection process, with the same UID, GID, and interface for `./bin/detect`. However, -- `/bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a /bin/detect that exits with a 0 exit code passes detection, and fails otherwise. -- If an extension is missing `/bin/detect`, the extension root `./detect` directory is treated as a pre-populated output directory (i.e., extensions can include a static build plan). +- `./bin/detect` is optional for extensions, and they are assumed to pass detection when it is not present. Just like with buildpacks, a `./bin/detect` that exits with a 0 exit code passes detection, and fails otherwise. +- If an extension is missing `./bin/detect`, the extension root `./detect` directory is treated as a pre-populated output directory (i.e., extensions can include a static build plan). - Extensions may only output `provides` entries to the build plan. They must not output `requires`. - Extensions are not included in `order` definitions (e.g., in `builder.toml`); instead, a separate `order-extensions` table should be used. The `order-extensions` table will be prepended to each group in the provided `order` (as if `order-extensions` were a composite buildpack). - Extensions are always `optional`. Extensions generate Dockerfiles before the regular buildpack build phase. -To generate these Dockerfiles, the lifecycle executes the extension's `/bin/generate` executable with the same UID, GID, and interface as regular buildpacks. +To generate these Dockerfiles, the lifecycle executes the extension's `./bin/generate` executable with the same UID, GID, and interface as regular buildpacks. However, -- Extensions `/bin/generate` must not write to the app directory. +- Extensions `./bin/generate` must not write to the app directory. - Extensions `` directory is replaced by an `` directory. -- If an extension is missing `/bin/generate`, the extension root `./generate` directory is treated as a pre-populated `` directory. +- If an extension is missing `./bin/generate`, the extension root `./generate` directory is treated as a pre-populated `` directory. -After `/bin/generate` executes, the `` directory may contain +After `./bin/generate` executes, the `` directory may contain - `build.toml`, with the same contents as a normal buildpack's `build.toml` (the `unmet` table array), but - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` - `run.toml`, @@ -97,7 +97,7 @@ Support for other instruction formats, e.g., LLB JSON files, could be added in t `build.Dockerfile` and `run.Dockerfile`target the builder image or runtime base image, respectively. -If no Dockerfiles are present, `/bin/generate` may still consume build plan entries. +If no Dockerfiles are present, `./bin/generate` may still consume build plan entries. Dockerfiles are applied to their corresponding base images after all extensions are executed and before any regular buildpacks are executed. Dockerfiles are applied in the order determined during buildpack detection. When multiple Dockerfiles are applied, the intermediate image generated from the application of the current Dockerfile will be provided as the `base_image` ARG to the next Dockerfile. Dockerfiles that target the run image (only) may ignore the provided `base_image` (e.g., `FROM some-other-image`). Dockerfiles that change the runtime base image may still use `COPY --from=${base_image}`. From 09ce1b12592ea51e96f8c5f3213db3ccabfcf8b7 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 25 Mar 2022 07:47:49 +0000 Subject: [PATCH 134/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Terence Lee Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 2a8be7891..2377bdc40 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -22,7 +22,7 @@ The `pack buildpack new` subcommand creates a new buildpack based on a shell-scr The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal `libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. -Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--target URL` allows other projects to re-use `pack` as their scaffolding tool. +Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--template URL` allows other projects to re-use `pack` as their scaffolding tool. `pack buildpack new --libcnb` supports end-users who wish to scaffold a new `libcnb`-based buildpack. The operation of `pack buildpack new --libcnb` does not require internet access to succeed. `pack buildpack new --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. From 625ad2da9be5fcca7b6a98570842f68c1ae2f7cd Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 6 May 2022 10:38:54 +0100 Subject: [PATCH 135/372] Replace `buildpack new` with `buildpack create` We replace the existing functionality where variables are provided to `pack buildpack new` as flags with `pack buildpack create` which prompts end-users for inputs. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 148 ++++++++++--------- 1 file changed, 77 insertions(+), 71 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 2377bdc40..957192a51 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -7,10 +7,12 @@ - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) +- Supersedes: https://github.com/buildpacks/rfcs/blob/main/text/0077-pack-buildpack-create.md # Summary -The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal extends `pack buildpack new` with a `--libcnb` option and a `--template URL` option to allow alternatives to the shell-script template. The `--libcnb` option creates a new buildpack written in golang using `libcnb`. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. +The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack prompting the end-user to choose an implementation language. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. + +We will implement the project scaffolding logic in a Go module separate from `pack`. # Definitions [definitions]: #definitions @@ -20,22 +22,18 @@ The `pack buildpack new` subcommand creates a new buildpack based on a shell-scr # Motivation [motivation]: #motivation -The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal `libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. - -Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--template URL` allows other projects to re-use `pack` as their scaffolding tool. +The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal Go/`libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. -`pack buildpack new --libcnb` supports end-users who wish to scaffold a new `libcnb`-based buildpack. The operation of `pack buildpack new --libcnb` does not require internet access to succeed. `pack buildpack new --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. +`pack buildpack create` supports end-users who wish to scaffold a new bash or Go/`libcnb`-based buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. -Extending `pack buildpack new` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. +Replacing `pack buildpack new` with `pack buildpack create` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. # What it is [what-it-is]: #what-it-is Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](https://yeoman.io/) [in](https://github.com/kowainik/summoner) [other](https://crates.io/crates/cargo-scaffold) [ecosystems](https://github.com/golang-standards/project-layout). Scaffolding systems are employed to ease onboarding of new developers. Within `pack` this feature is targeted at onboarding buildpack authors. -- Explaining the feature largely in terms of examples. - -`pack buildpack new --libcnb` should generate skeleton code according to the [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: +`pack buildpack create` should prompt the end user to choose an implementation language. Thereafter, generate skeleton code according to the best practices for that language. For example, users choosing Go as an implementation language would generate a [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: ```bash . @@ -49,50 +47,79 @@ Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](ht └── detect.go ``` -This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the `--libcnb` skeleton. `pack buildpack new --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with `--libcnb` onboarding and support experienced buildpack authors to use the scaffolding of their choice. +This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the generated project. `pack buildpack create --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with onboarding and support experienced buildpack authors to use the scaffolding of their choice. # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from an embedded file system. +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. -A full invocation to generate `libcnb` scaffolding is similar to the [documented project scaffolding](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). +A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). ```bash -pack buildpack new --libcnb examples/ruby \ - --api 0.7 \ - --path ruby-buildpack \ - --version 0.0.1 \ - --stacks io.buildpacks.samples.stacks.bionic +pack buildpack new +Use the arrow keys to navigate: ↓ ↑ → ← +? Choose a project template: + ▸ bash + go + ``` A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```bash -pack buildpack new --libcnb examples/ruby \ - --api 0.7 \ - --path ruby-buildpack \ - --version 0.0.1 \ - --stacks io.buildpacks.samples.stacks.bionic -Package name (often github.com/username/repo)? github.com/AidanDelaney/ruby -Created project in ruby-buildpack +pack buildpack new +Use the arrow keys to navigate: ↓ ↑ → ← +? Choose a project template: + ▸ bash + go +✔ Enter a directory in which to scaffold the project: bash_buildpack +Use the arrow keys to navigate: ↓ ↑ → ← +? Choose the buildpack API version (use the default if you are unsure): + ▸ 0.7 + 0.8 +✔ Enter an identifier for the buildpack: example/bash +✔ Enter the initial buildpack version: 0.0.1 +✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic + +Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.yaml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and golang skeleton code is: +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and Go skeleton code is: ```bash . -├── prompts.yaml -├── cmd - └── main.go +├── prompts.toml +└── {{.ProjectDirectory}} + ├── buildpack.toml + ├── cmd + │   └── main.go + ├── go.mod + └── pkg + ├── build.go + └── detect.go ``` -Where `prompts.yaml` is of the form: +Where `prompts.toml` is of the form: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +default="bash_buildpack" +required=true + +[[prompt]] +name="BuildpackApi" +prompt="Choose the buildpack API version (use the default if you are unsure)" +choices=["0.7", "0.8"] -```yaml -prompts: - - variable: packageName - prompt: Package name (often github.com/username/repo) +[[prompt]] +name="ModuleName" +prompt="Enter a valid Go module name for this buildpack" +default="github.com/user/buildpack" + +... ``` And an example templated source file is: @@ -103,7 +130,7 @@ package main import ( "github.com/buildpacks/libcnb" - bp "{{ .packageName }}/pkg" + bp "{{ .ModuleName }}/pkg" ) func main() { @@ -114,39 +141,20 @@ func main() { } ``` -The following template variables, known as the _reserved template variables_ are available for substitution in all project files: - -* `.ID` -- the buildpack id that will be written to `buildpack.toml` obtained from a positional argument to `pack` -* `.Version` -- the buildpack version obtained from the argument to the `--version` flag -* `.API` -- the buildpack API version obtained from the argument to the `--api` flag -* `.Stacks` -- a slice containing the buildpack stacks obtained from the argument to the `--stacks` flag - -Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file and not redefine reserved template variables. An optional default value may be provided. Variables that do not provide a default value must +Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. -```yaml -prompts: - - variable: packageName - prompt: Package name (often github.com/username/repo) - default: DefaultValue -``` - -Answers to prompts can be provided in a yaml file. This supports programmatic use of `pack`: +Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`: ``` -pack buildpack new --libcnb examples/ruby \ - --config-file answers.yaml \ - --api 0.7 \ - --path ruby-buildpack \ - --version 0.0.1 \ - --stacks io.buildpacks.samples.stacks.bionic -Created project in ruby-buildpack +pack buildpack create --overrides overrides.toml +Created project in bash_buildpack ``` -The format of `--config-file` parallels the format used in `cookiecutter`. An example of which is: +The format of `--overrides` parallels the `prompts.toml` format. An example of which is: -```yaml -defaultContext: - packageName: com.example/an-example +```toml +ProjectDirectory="test_example" +ModuleName="github.com/test/test" ``` All input files are expected to be normalized to Unix line endings. @@ -154,7 +162,7 @@ All input files are expected to be normalized to Unix line endings. # Migration [migration]: #migration -The current `bash` project scaffolding can be embedded in `pack` as an embedded file system. This would allow the `--libcnb` and `--target URL` code to be reused. +The current `bash` project scaffolding can be re-used in the default project scaffolding. # Drawbacks [drawbacks]: #drawbacks @@ -162,28 +170,26 @@ The current `bash` project scaffolding can be embedded in `pack` as an embedded Why should we *not* do this? * Golang project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. -* Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpaks/bash my_buildpack`. -* Including `--libcnb` project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of the embedded file system plus the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). The `pack` binary will grow to +* Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpacks/bash my_buildpack`. +* Including generalized project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). * This proposal commits to support a specific project scaffolding format. A migration path should be established if and when a de-facto standard golang template library becomes available. -* In supporting `--libcnb` we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. +* In default project scaffolding we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. # Alternatives [alternatives]: #alternatives - What other designs have been considered? -We have considered a wider re-implementation of [cookiecutter](https://cookiecutter.readthedocs.io/). However, the scope of a re-implementation was considered too wide. In addition, we have considered a cookiecutter-lite implementation where project scaffolding is cloned from a repository. This proposal recommends a project scaffolding that can be embedded in `pack` and, in addition, can be downloaded from an Internet source. - We have also considered [springerle](https://github.com/carlmjohnson/springerle) which uses a single txtar file to describe skeleton project structure. The use of a single txtar file requires template providers to write and maintain this format. [bare](https://github.com/bare-cli/bare) is a new project. It assumes that all templates are stored on github.com. - Why is this proposal the best? -To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that embeds cnb-provided skeletons in `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. +To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that integrates cnb-provided skeletons with `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. - What is the impact of not doing this? -Omitting support for `libcnb` project scaffolding requires new buildpack authors to consult our documentation about project structure. Moreover, as `pack` supports scaffolding of shell-script buildpacks the impression is given that the buildpacks project _prefers_ shell implementations. +Omitting support for generalized project scaffolding requires new buildpack authors to consult our documentation about project structure. Moreover, as `pack` supports scaffolding of shell-script buildpacks the impression is given that the buildpacks project _prefers_ shell implementations. # Prior Art [prior-art]: #prior-art @@ -192,7 +198,7 @@ There are many, many competing implementations of project scaffolding tools. * Python's [Cookiecutter](https://cookiecutter.readthedocs.io/): widely used to scaffold projects in many languages, including golang. Requires a Python runtime to be available. * [springerle](https://github.com/carlmjohnson/springerle): golang re-think of cookiecutter. Springerle uses a single txtar file augmented with a header containing user prompts. This proposal prefers using a filesystem rather than a single txtar file as the filesystem approach extends to cloning repositories. -* [cgapp](https://github.com/create-go-app/cli): This proposal heavily borrows from cgapp. It is necessary to modify parts of cgapp in order to apply variable substitution. +* [cgapp](https://github.com/create-go-app/cli): This proposal heavily borrows from cgapp. * JavaScript's [Yeoman](https://yeoman.io/): widely used in the web ecosystem * [boilr](https://github.com/tmrts/boilr/): golang cookiecutter. Moribund project. * [bare](https://github.com/bare-cli/bare): golang cookiecutter. Possible successor to boilr. Both boilr and bare assume the templates are stored on github.com and use the zip downloading functionality of that specific service. From cd03bda4f3864c57025e871412d744c032a6d51e Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 10 Jun 2022 06:36:24 +0100 Subject: [PATCH 136/372] Pass pack buildpack create overrides as CLI parameters Update the RFC so that overrides to buildpack creation variables can be passed on the command line. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 957192a51..19203a850 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -52,7 +52,7 @@ This approach is not opinionated on topics such as which testing framework to us # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). @@ -146,17 +146,10 @@ Template variables introduced in `prompts.yaml` are required to have a name uniq Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`: ``` -pack buildpack create --overrides overrides.toml +pack buildpack create --override 'ProjectDirectory="test_example"' --overrride='ModuleName="github.com/test/test"' Created project in bash_buildpack ``` -The format of `--overrides` parallels the `prompts.toml` format. An example of which is: - -```toml -ProjectDirectory="test_example" -ModuleName="github.com/test/test" -``` - All input files are expected to be normalized to Unix line endings. # Migration From fa16becf92982bf1e856a8505cd9b6c767b7922c Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 10 Jun 2022 06:49:44 +0100 Subject: [PATCH 137/372] Modify list indicators Consistently use '-' to denote a list item Co-authored-by: Javier Romero Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 19203a850..628fae5d7 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -199,9 +199,6 @@ There are many, many competing implementations of project scaffolding tools. # Unresolved Questions [unresolved-questions]: #unresolved-questions -- What parts of the design do you expect to be resolved before this gets merged? -- What parts of the design do you expect to be resolved through implementation of the feature? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From 23648efec0bde8a44e8f0ea03f458c5a17480360 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 14 Jun 2022 08:51:43 +0100 Subject: [PATCH 138/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Daniel Mikusa Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 628fae5d7..cbb2d2fc6 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -52,7 +52,7 @@ This approach is not opinionated on topics such as which testing framework to us # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). From 2db89b98bdd93618415bdc2b387576660b4d4ee2 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 14 Jun 2022 08:51:52 +0100 Subject: [PATCH 139/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Daniel Mikusa Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index cbb2d2fc6..b7a3aa261 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -57,7 +57,7 @@ The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1 A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). ```bash -pack buildpack new +pack buildpack create Use the arrow keys to navigate: ↓ ↑ → ← ? Choose a project template: ▸ bash From 5a429decd38db7beedfcce1c925f6a0d8469d42f Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 14 Jun 2022 08:51:59 +0100 Subject: [PATCH 140/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Daniel Mikusa Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index b7a3aa261..3ceccc39b 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -68,7 +68,7 @@ Use the arrow keys to navigate: ↓ ↑ → ← A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```bash -pack buildpack new +pack buildpack create Use the arrow keys to navigate: ↓ ↑ → ← ? Choose a project template: ▸ bash From daa156fa13a0a6a5a63b6126d8eacd93e289f2c5 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Wed, 29 Jun 2022 09:55:39 +0100 Subject: [PATCH 141/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Javier Romero Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 3ceccc39b..3eab7b956 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -198,7 +198,10 @@ There are many, many competing implementations of project scaffolding tools. # Unresolved Questions [unresolved-questions]: #unresolved-questions - +* Where do the template repositories live for `bash` and `golang`? + * +* What team maintains template repositories? + * Buildpack Author Tooling team # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From 1dede4c92caabcf11bbe30157375ee8dd4df7bee Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Wed, 29 Jun 2022 09:51:59 +0100 Subject: [PATCH 142/372] Reduce the proposal scope The proposal scope is reduced to providing a more generic mechanism to scaffold buildpacks written in bash. The provision of libcnb project scaffolding will be the focus of a subsequent RFC. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 95 +++++--------------- 1 file changed, 24 insertions(+), 71 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 3eab7b956..8b87c8890 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -10,7 +10,7 @@ - Supersedes: https://github.com/buildpacks/rfcs/blob/main/text/0077-pack-buildpack-create.md # Summary -The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack prompting the end-user to choose an implementation language. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. +The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack using the existing `bash` scaffolding. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. We will implement the project scaffolding logic in a Go module separate from `pack`. @@ -22,9 +22,9 @@ We will implement the project scaffolding logic in a Go module separate from `pa # Motivation [motivation]: #motivation -The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal Go/`libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. +The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a mechanism to `pack` that would allow the buildpacks project to provide `libcnb` buildpack project scaffolding. -`pack buildpack create` supports end-users who wish to scaffold a new bash or Go/`libcnb`-based buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. +`pack buildpack create` supports end-users who wish to scaffold a new bash buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. Replacing `pack buildpack new` with `pack buildpack create` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. @@ -33,71 +33,38 @@ Replacing `pack buildpack new` with `pack buildpack create` allows buildpack aut Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](https://yeoman.io/) [in](https://github.com/kowainik/summoner) [other](https://crates.io/crates/cargo-scaffold) [ecosystems](https://github.com/golang-standards/project-layout). Scaffolding systems are employed to ease onboarding of new developers. Within `pack` this feature is targeted at onboarding buildpack authors. -`pack buildpack create` should prompt the end user to choose an implementation language. Thereafter, generate skeleton code according to the best practices for that language. For example, users choosing Go as an implementation language would generate a [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: - -```bash -. -├── buildpack.toml -├── cmd -│ └── main.go -├── go.mod -├── go.sum -└── pkg - ├── build.go - └── detect.go -``` - -This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the generated project. `pack buildpack create --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with onboarding and support experienced buildpack authors to use the scaffolding of their choice. +`pack buildpack create` creates the same buildpack scaffolding as `pack buildpack new`. However, `pack buildpack create --template URL` can be used to access alternative project scaffolding. # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. - -A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell scaffolding from a cloned git repoistory. -```bash -pack buildpack create -Use the arrow keys to navigate: ↓ ↑ → ← -? Choose a project template: - ▸ bash - go - -``` +The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--stacks` command line flags. These command line flags will be preserved in `pack buildpack create`. However, where a flag is omitted, the end-user will be prompted to provide an appropriate value. A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```bash -pack buildpack create -Use the arrow keys to navigate: ↓ ↑ → ← -? Choose a project template: - ▸ bash - go +pack buildpack create example/bash ✔ Enter a directory in which to scaffold the project: bash_buildpack Use the arrow keys to navigate: ↓ ↑ → ← ? Choose the buildpack API version (use the default if you are unsure): ▸ 0.7 0.8 -✔ Enter an identifier for the buildpack: example/bash -✔ Enter the initial buildpack version: 0.0.1 ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and Go skeleton code is: +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as.: ```bash . ├── prompts.toml └── {{.ProjectDirectory}} - ├── buildpack.toml - ├── cmd - │   └── main.go - ├── go.mod - └── pkg - ├── build.go - └── detect.go + └── bin + ├── build + └── detect ``` Where `prompts.toml` is of the form: @@ -114,39 +81,24 @@ name="BuildpackApi" prompt="Choose the buildpack API version (use the default if you are unsure)" choices=["0.7", "0.8"] -[[prompt]] -name="ModuleName" -prompt="Enter a valid Go module name for this buildpack" -default="github.com/user/buildpack" - ... ``` -And an example templated source file is: - -```golang -package main - -import ( - "github.com/buildpacks/libcnb" +Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. - bp "{{ .ModuleName }}/pkg" -) +Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. -func main() { - libcnb.Main( - bp.Detect{}, - bp.Build{}, - ) -} +``` +pack buildpack create --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' +✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic +Created project in bash_buildpack ``` -Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. - -Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`: +The above overrides are equivalent to: ``` -pack buildpack create --override 'ProjectDirectory="test_example"' --overrride='ModuleName="github.com/test/test"' +pack buildpack create --path test_example --api 0.8 +✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` @@ -157,16 +109,17 @@ All input files are expected to be normalized to Unix line endings. The current `bash` project scaffolding can be re-used in the default project scaffolding. +We intend to maintain `pack buildpack new` in parallel with `pack buildpack create` for two `pack` releases. After two `pack` releases, `pack buildpack new` will be replaced with an error instructing the user to run `pack buildpack create`. We will remove `pack buildpack new` after three `pack` releases. + # Drawbacks [drawbacks]: #drawbacks Why should we *not* do this? -* Golang project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. +* Project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. * Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpacks/bash my_buildpack`. * Including generalized project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). * This proposal commits to support a specific project scaffolding format. A migration path should be established if and when a de-facto standard golang template library becomes available. -* In default project scaffolding we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. # Alternatives [alternatives]: #alternatives @@ -178,7 +131,7 @@ We have also considered [springerle](https://github.com/carlmjohnson/springerle) [bare](https://github.com/bare-cli/bare) is a new project. It assumes that all templates are stored on github.com. - Why is this proposal the best? -To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that integrates cnb-provided skeletons with `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. +To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that integrates future cnb-provided project scaffolding with `pack` and allows `pack` to clone 3rd party project scaffolding from a location chosen by the end-user. - What is the impact of not doing this? From bd4d9883b0d6922e24c282dd4a2e49abe9663b01 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Wed, 29 Jun 2022 10:03:21 +0100 Subject: [PATCH 143/372] Add buildpack id to examples Provide more complete CLI examples Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 8b87c8890..09f3899c1 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -89,7 +89,7 @@ Template variables introduced in `prompts.yaml` are required to have a name uniq Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. ``` -pack buildpack create --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' +pack buildpack create example/bash --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` @@ -97,7 +97,7 @@ Created project in bash_buildpack The above overrides are equivalent to: ``` -pack buildpack create --path test_example --api 0.8 +pack buildpack create example/bash --path test_example --api 0.8 ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` From c71e7fea905a9872d53e9c7ecd782650934e77b5 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Thu, 14 Jul 2022 09:13:46 +0100 Subject: [PATCH 144/372] Provide fuller specification of prompt.toml Better specify prompts.toml and introduce both templates and template collections by example. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 172 +++++++++++++++++-- 1 file changed, 159 insertions(+), 13 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 09f3899c1..17b3ee142 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -12,7 +12,7 @@ # Summary The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack using the existing `bash` scaffolding. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. -We will implement the project scaffolding logic in a Go module separate from `pack`. +We will implement the project scaffolding logic in a Go module separate from `pack`. This decouples `pack` from the project scaffolding implementation. # Definitions [definitions]: #definitions @@ -24,7 +24,7 @@ We will implement the project scaffolding logic in a Go module separate from `pa The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a mechanism to `pack` that would allow the buildpacks project to provide `libcnb` buildpack project scaffolding. -`pack buildpack create` supports end-users who wish to scaffold a new bash buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. +`pack buildpack create` supports end-users who wish to scaffold a new buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. Replacing `pack buildpack new` with `pack buildpack create` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. @@ -44,8 +44,8 @@ The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--st A full session includes the terminal prompts that the project scaffolding tool asks of the end user: -```bash -pack buildpack create example/bash +```command +$ pack buildpack create example/bash ✔ Enter a directory in which to scaffold the project: bash_buildpack Use the arrow keys to navigate: ↓ ↑ → ← ? Choose the buildpack API version (use the default if you are unsure): @@ -56,7 +56,16 @@ Use the arrow keys to navigate: ↓ ↑ → ← Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as.: +The user can skip common prompts by providing `--api`, `--path` or `--stacks` as command line flags. Running the same example as above, but providing both `--api` and `--stacks`, but not `--path`, results in the user being prompted only to provide the output directory. We provide more detail on how flags override prompts in a subsection below. + +```command +$ pack buildpack create example/bash --api 0.8 --stacks io.buildpacks.samples.stacks.bionic +✔ Enter a directory in which to scaffold the project: bash_buildpack + +Created project in bash_buildpack +``` + +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as follows, a full specification of `prompts.toml` is provided in the following subsection. ```bash . @@ -84,25 +93,162 @@ choices=["0.7", "0.8"] ... ``` -Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. - -Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. +Answers to prompts can be provided in an `overrides.toml` file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. -``` -pack buildpack create example/bash --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' +```command +$ pack buildpack create example/bash --override='ProjectDirectory="test_example"' --override='BuildpackApi=0.8' ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` The above overrides are equivalent to: -``` -pack buildpack create example/bash --path test_example --api 0.8 +```command +$ pack buildpack create example/bash --path test_example --api 0.8 ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` -All input files are expected to be normalized to Unix line endings. +Multiple templates can be managed in a single repository, referred to as a **template colleciton**. + +## Template Collections + +We expect project templates to be managed in `git` repositories. Projects may wish to manage multiple templates in a single repository. We refer to this as a template collection. More formally, a template collection is a `git` repository where top-level directories are templates. A template collection does not contain a top-level `prompts.toml`, but top-level subdirectories are expected to contain `prompts.toml` files. As an example, a template collection might have the following structure where the `bash` sub-directory is a template and the `Go` sub-directory is a template. + +```bash +. +├── bash +└── Go +``` + +A user may use a template collection as an argument to `--template`. A template collection passed as `--template` first prompts the end user to choose between the available templates in the repository. Once a template is chosen, the user is prompted using the `prompts.toml` from the chosen template. + +```command +$ pack buildpack create --template https://github.com/AidanDelaney/cnb-buildpack-templates +Use the arrow keys to navigate: ↓ ↑ → ← +? choose a project template: + ▸ Go + bash +Enter a directory in which to scaffold the project: go_buildpack +? Choose the buildpack API version (use the default if you are unsure): + 0.7 + ▸ 0.8 +✔ 0.8 +Enter an identifier for the buildpack: example/golang +Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic +✔ Enter a valid Go module name for this buildpack: github.com/user/buildpack +Created project in go_buildpack +``` + +A user may specify the choice of template from a template collection via a `--sub-path` flag. A user may choose a `bash` template from a template collection and provide the api, output directory and stacks as command line flags: + +```command +$ pack buildpack create example/golang --template https://github.com/AidanDelaney/cnb-buildpack-templates \ + --sub-path Go \ + --api 0.8 \ + --path go_buildpack \ + --stacks io.buildpacks.samples.stacks.bionic +Created project in go_buildpack +``` + +Having provided examples of the use of `pack buildpack create`. We now specify the format of `prompts.toml`. + +## prompts.toml + +The prompts are contained in a TOML file. The `prompts.toml` file contains an array of tables. Each table specifies a single `prompt`. Each prompt defines a variable identified by a provided `name`. + +| field | required | description | type | +|---------------|----------|------------------------------------------------------------------------------------------------------------------------------------------|----------------| +| name | Yes | The variable identifier. Must be unique within the `prompts.toml` file. | string | +| prompt | Yes | The default prompt question to ask the user | string | +| prompt.locale | No | A translation of the default prompt into the locale specified. | | +| required | No | Evaluates to `false` if not provided. Specifies whether the user **must** provide a value for the variable identified by `name`. | boolean | +| default | No | A default value for the variable identified by `name`. Mutually exclusive to `choices` | string | +| choices | No | A list of default values from which the user may choose a value for the variable identified by `name`. Mutually exclusive to `default`. | list of stings | + +Prompt locales are specified following [IETF BCP 47](https://tools.ietf.org/rfc/bcp/bcp47.txt). As such `prompt.en-US` is the translation of `prompt` into US English and `prompt.es-MX` is the translation of `prompt` into Mexican Spanish. + +For example, the following is a valid `prompts.toml` file: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +``` + +The following `prompts.toml` is invalid as it does not contain the required `name` field: + +```toml +[[prompt]] +prompt="Enter a directory in which to scaffold the project" +``` + +The following `prompts.toml` is invalid as value of the `name` field is not unique within the document: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" + +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory" +``` + +The following `prompts.toml` is invalid as the `default` and `choices` fields are mutually excusive: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +default="/tmp" +choices=["/tmp", "/home"] +``` + +### Overrides + +If a template contains both a `prompts.toml` file and an `overrides.toml` file, then the `overrides.toml` file provides values for variables introduced in `prompts.toml`. + +Given a `prompts.toml` file + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +``` + +And an `overrides.toml` file + +```toml +ProjectDirectory="/tmp" +``` + +Then the value "/tmp" is interpreted as the value of the variable identified by `ProjectDirectory`. + + +Overrides may also be provided as command line arguments. For example `pack buildpack create --overrides ProjectDirectory=/tmp` provides a value, `/tmp`, for the variable identified by the name `ProjectDirectory`. + +Overrides provided as command line flags have **higher precedence** than overrides provided in `overrides.toml`. + +When an override is provided `pack buildpack create` must not prompt the user for overriden values. + +## `pack buildpack create` flags as prompts variables + +If specified, the following `pack buildpack create` flags override the value of variables with the listed identifier: + +| command line flag | variable name | +|-------------------|--------------------| +| `--path` | `ProjectDirectory` | +| `--api` | `BuildpackApi` | +| `--stacks` | `BuildpackStacks` | + +In addition, the required positional argument to `pack buildpack create`, eg: `pack buildpack create example/bash`, overrides the variable identified by `BuildpackID`. + +## `pack buildpack create` Substitutions + +The operation of variable substitution follows the operation of Go [`text/template`](https://pkg.go.dev/text/template). Prompts defined in `prompts.toml` are interpreted, the user may be prompted and set of variables is generated. The identifiers of variables are replaced with the value of the variable. + +Variables may be used in files, for example where a `prompts.toml` defines a variable with identifier `ProjectDirectory` then the expression `{{.ProjectDirectory}}` in files is replaced with the value of the variable identified by `ProjectDirectory`. In addition, the expression `{{.ProjectDirectory}}` may be used in template file paths. The generated project substitutes file paths with variable expressions with the value of the variable. # Migration [migration]: #migration From 9325a131940ad5613bc85b4027b26f44e18da021 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 14 Jul 2022 15:22:16 -0400 Subject: [PATCH 145/372] Updates per 7/14 Working Group Signed-off-by: Natalie Arellano --- text/0000-dockerfiles.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/text/0000-dockerfiles.md b/text/0000-dockerfiles.md index 23e9eebd7..873a6e3f9 100644 --- a/text/0000-dockerfiles.md +++ b/text/0000-dockerfiles.md @@ -87,8 +87,8 @@ However, - If an extension is missing `./bin/generate`, the extension root `./generate` directory is treated as a pre-populated `` directory. After `./bin/generate` executes, the `` directory may contain -- `build.toml`, with the same contents as a normal buildpack's `build.toml` (the `unmet` table array), but - - With an additional `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` +- `build.toml`, + - With an `args` table array with `name` and `value` fields that are provided as build args to `build.Dockerfile` - `run.toml`, - With an `args` table array with `name` and `value` fields that are provided as build args to `run.Dockerfile` - Either or both of `build.Dockerfile` and `run.Dockerfile` @@ -97,7 +97,7 @@ Support for other instruction formats, e.g., LLB JSON files, could be added in t `build.Dockerfile` and `run.Dockerfile`target the builder image or runtime base image, respectively. -If no Dockerfiles are present, `./bin/generate` may still consume build plan entries. +If no Dockerfiles are present, `./bin/generate` may still consume build plan entries. Unlike buildpacks, extensions must consume all entries in the provided plan (they cannot designate any entries as "unmet"). Dockerfiles are applied to their corresponding base images after all extensions are executed and before any regular buildpacks are executed. Dockerfiles are applied in the order determined during buildpack detection. When multiple Dockerfiles are applied, the intermediate image generated from the application of the current Dockerfile will be provided as the `base_image` ARG to the next Dockerfile. Dockerfiles that target the run image (only) may ignore the provided `base_image` (e.g., `FROM some-other-image`). Dockerfiles that change the runtime base image may still use `COPY --from=${base_image}`. @@ -137,7 +137,6 @@ Note: The Dockerfiles referenced must disable rebasing, and build times will be [ -f package.json ] && cp "$CNB_BUILDPACK_DIR/Dockerfile-node" "$1/build.Dockerfile" ``` - ### Dockerfiles for Base Images The same Dockerfile format may be used to create new base images or modify existing base images outside of the app build process (e.g., before creating a builder). Any specified labels override existing values. From 266bfb01ce288954d59489a89ef43767371f9e30 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 20 Jul 2022 16:49:27 -0400 Subject: [PATCH 146/372] Fill in motivation section Signed-off-by: Emily Casey --- text/0000-rfc-process.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index 67c78d606..a53bd2334 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -69,7 +69,11 @@ The RFC process **is not**: This RFC process is an evolution of our [previous process](0004-rfc-process.md). -TODO +As the project grows our process needs to grow with it. Compared to the previous iteration, this new process should: +1. **Re-balance responsibilities**. We wish to empower team leads to take ownership of the future of the project and transition the TOC into an oversight role. +2. **Favor action**. By moving from a super-majority vote to a lazy-consensus we remove the ability to block change through inaction. It is now the responsibility of those with concerns to raise them in a timely fashion. +3. **Manage our bandwidth**. Sometimes we need to say "not now" to a good idea to manage the number of large changes proceeding simultaneously. This new process should help the project focus on shipping high-quality enhancements by managing the number of in-flight RFCs, and ensuring each approved RFC has an implementer. +4. **Improve Readability**. By allowing amendments to RFCs we make life easier for anyone wishing to understand a change by ensuring they can find the full plan in a single document. # What it is [what-it-is]: #what-it-is From 7bf9865c33df1036625cb2a9127282d62eb5ade3 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 20 Jul 2022 16:50:07 -0400 Subject: [PATCH 147/372] minor fixes --- text/0000-rfc-process.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index a53bd2334..091449228 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -95,12 +95,12 @@ If there is any doubt, maintainers should prefer opening an "unecessary" RFC to An RFC should be a project RFC if: * it impacts the spec. * it introduces a new component. -* its implementation necessitates coordination across multiple team. +* its implementation necessitates coordination across multiple teams. * it meaningfully impacts multiple personas (e.g. buildpack authors and platform authors). * the TOC requests that it be a project RFC. Given the nature of our project many RFCs will happen at the project level. However, some types of changes are more appropriately scoped to the team level. Some examples include: -* Platform example: additions to or modification of the pack CLI interface (e.g. [pack pull policy](0046-pack-pull-policy.md)) or library interface (e.g. [pack logging refactor](0002-pack-logging-refactor.md)), provided these changes do not require changes to components external to the platform team. +* Platform example: additions to or modification of the pack CLI interface (e.g. [pack pull policy](0046-pack-pull-policy.md)) or library interface (e.g. [pack logging refactor](0002-pack-logging-refactor.md)). * BAT example: a new major version of the libcnb API. * Implementation example: [Layer history](0102-history.md) or the lifecycle [multicall binary](0024-lifecycle-multicall-binary-for-build.md). * Distribution example: [Update CNB Registry JSON Schema](https://github.com/buildpacks/rfcs/pull/45). From 9bd4f396adaa60ad1c02ea46729953f74cd69509 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Wed, 20 Jul 2022 16:51:58 -0400 Subject: [PATCH 148/372] Adds Requesting an RFC seciton to Process Signed-off-by: Emily Casey --- text/0000-rfc-process.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index 091449228..5ab6f7ece 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -108,7 +108,10 @@ Given the nature of our project many RFCs will happen at the project level. Howe ### Process -#### Drafting +#### Requesting an RFC +If you believe an RFC should be written but are either not prepared to author it personally or not prepared to do it _now_, please open an issue on this repo with a high level description of the desired change. This helps us keep track of good ideas and match make ambitious contributors with appropriately sized challenges. + +#### Drafting an RFC All RFCs begin life as a draft. Anyone wishing to propose a change to the project should create a draft RFC by: - Fork the RFC repo: From a31c7051e24b21e5ef3a45a97209f266508ebae2 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Thu, 28 Jul 2022 09:42:18 -0400 Subject: [PATCH 149/372] Adds ammendment mechanism Signed-off-by: Emily Casey --- 0000-template.md | 21 +++++++++++++++++++++ text/0000-rfc-process.md | 8 +++++++- 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/0000-template.md b/0000-template.md index b4256f4e7..cc5d06dc3 100644 --- a/0000-template.md +++ b/0000-template.md @@ -78,3 +78,24 @@ Discuss prior art, both the good and bad. Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. + +# History +[history]: #history + + \ No newline at end of file diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index 5ab6f7ece..02fcc011f 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -165,7 +165,13 @@ The same RFC may be re-opened and brought to a vote again in the future assuming ### Amending an RFC -TODO +While we should strive to get the details right the first time, by doing our due diligence including a proof of concept implementation for larger/riskier RFCs, there will be times when we discover during the process of implementation that something about the original plan was incomplete or needs adjustment. Small changes may be PR'ed by the implementer and merged upon approval by the steward. It is the responsibility of the Steward to determine what changes are minor enough to qualify as an amendment and which are fundamental and/or controversial enough to require a new superseding RFC. + +PRs that amend an RFC should add an amendment to the [History](../0000-template.md#history) section of the RFC document with: +1. Amendment Metadata +2. A summary of the changes +3. The motivation for the changes + ### Amending the RFC Process From 9afb723934c5b5a767ebe430ac51307123f1ffc9 Mon Sep 17 00:00:00 2001 From: Emily Casey Date: Thu, 28 Jul 2022 09:55:10 -0400 Subject: [PATCH 150/372] Documents Labels Signed-off-by: Emily Casey --- text/0000-rfc-process.md | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index 02fcc011f..8d6516305 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -163,7 +163,9 @@ If a single no vote is cast before the arrival of the end date the RFC is immedi The same RFC may be re-opened and brought to a vote again in the future assuming the concerns that lead to the no vote are addressed. -### Amending an RFC +##### Implementation + +##### Amending an RFC While we should strive to get the details right the first time, by doing our due diligence including a proof of concept implementation for larger/riskier RFCs, there will be times when we discover during the process of implementation that something about the original plan was incomplete or needs adjustment. Small changes may be PR'ed by the implementer and merged upon approval by the steward. It is the responsibility of the Steward to determine what changes are minor enough to qualify as an amendment and which are fundamental and/or controversial enough to require a new superseding RFC. @@ -180,9 +182,22 @@ The RFC process should be amended through the RFC process. However, the TOC rese # How it Works [how-it-works]: #how-it-works -### Amending the RFC Process - -The RFC process should be amended through the RFC process. However, the TOC reserves the right to change the process via a super-majority vote in the unlikely even that the process prove so irreparably flawed as to preclude its amendment via the process. +### Labels + +| name | purpose +|---------------------------|-------- +| `type/idea` | An Issues requesting an RFC +| `type/rfc` | A PR containing a new RFC +| `type/tracking` | An issue tracking the implementation of an RFC +| `type/ammendment` | A PR containing an ammendment to an existing RFC +| `scope/project` | Applied to project-level RFCs +| `scope/team` | Applied to team-level RFCs. +| `team/` | E.g `team/platform`. Designates the team that "owns" an RFC. Applies to both team-level and project-level RFCs. For project-level RFCs the lead of this team is the steward. +| `status/needs-steward` | Applied to all new RFCs. Removed once the RFC is accepted by a steward or closed. +| `status/ready-for-review` | The Steward applies this to an RFC when they deem it ready for review by a broader audience. +| `status/voting` | The Steward applies this label when the voting window opens +| `status/accepted` | The Steward applies this label to accepted RFCs. +| `status/rejected` | The Steard applies this label to a rejected RFCs. # Migration [migration]: #migration From bc3c4af4939b3479a831c7c355c2b3478f1297b6 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 10 Aug 2022 15:46:41 -0400 Subject: [PATCH 151/372] Updates op command for newer versions of op - Uses `op read` instead of `op get` Signed-off-by: Natalie Arellano --- merge-rfc.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/merge-rfc.sh b/merge-rfc.sh index 54c9e1b0f..8f90848f5 100755 --- a/merge-rfc.sh +++ b/merge-rfc.sh @@ -55,7 +55,7 @@ require_command issues-generation if [[ -z "${GITHUB_TOKEN:-}" ]]; then require_command op echo "> Pulling GitHub token from vault..." - GITHUB_TOKEN=$(op get item 7xorpxvz3je3vozqg3fy3wrcg4 --vault "Shared" --account buildpacks | jq -r '.details.sections[] | select(.fields).fields[] | select(.t == "credential").v') + GITHUB_TOKEN=$(op read op://Shared/7xorpxvz3je3vozqg3fy3wrcg4/credential --account buildpacks) fi #### From 465edc2505b821097207ffec6079cfe230ca3647 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 10 Aug 2022 15:47:59 -0400 Subject: [PATCH 152/372] RFC 0105 [#173] Signed-off-by: Natalie Arellano --- text/{0000-dockerfiles.md => 0105-dockerfiles.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename text/{0000-dockerfiles.md => 0105-dockerfiles.md} (95%) diff --git a/text/0000-dockerfiles.md b/text/0105-dockerfiles.md similarity index 95% rename from text/0000-dockerfiles.md rename to text/0105-dockerfiles.md index 873a6e3f9..ceb98f394 100644 --- a/text/0000-dockerfiles.md +++ b/text/0105-dockerfiles.md @@ -3,9 +3,9 @@ - Name: Support Dockerfiles - Start Date: 2021-06-30 - Author(s): sclevine -- RFC Pull Request: (leave blank) +- RFC Pull Request: [rfcs#173](https://github.com/buildpacks/rfcs/pull/173) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/lifecycle#890](https://github.com/buildpacks/lifecycle/issues/890), [buildpacks/lifecycle#891](https://github.com/buildpacks/lifecycle/issues/891), [buildpacks/lifecycle#892](https://github.com/buildpacks/lifecycle/issues/892), [buildpacks/lifecycle#893](https://github.com/buildpacks/lifecycle/issues/893), [buildpacks/lifecycle#894](https://github.com/buildpacks/lifecycle/issues/894) - Supersedes: [RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md), [RFC#167](https://github.com/buildpacks/rfcs/pull/167) - Depends on: [RFC#172](https://github.com/buildpacks/rfcs/pull/172) From b5b778bb96e5e5d612fab7bab516d6b219d319aa Mon Sep 17 00:00:00 2001 From: Andrew Gracey Date: Mon, 15 Aug 2022 20:13:20 -0700 Subject: [PATCH 153/372] clarifying a few points, adding a 3rd use-case, and changing toml file --- text/0000-additonal-oci-artifacts.md | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md index be099efa8..ccf1f6fb0 100644 --- a/text/0000-additonal-oci-artifacts.md +++ b/text/0000-additonal-oci-artifacts.md @@ -31,33 +31,32 @@ Cloud Native Buildpacks are awesome for building container images but come with # What it is [what-it-is]: #what-it-is -This change allows for additional flexibility in what buildpack authors and platforms can build on top of buildpacks. Two initial use cases are WebAssembly and Helm. +This change allows for additional flexibility in what buildpack authors and platforms can build on top of buildpacks. Three initial use cases are WebAssembly, test output, and Helm charts. For the first, a buildpack would be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or runwasi). Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, ConfigMaps to pass around out-of-band, etc...) +Lastly, it may be advantageous to allow a buildpack to produce the results of unit tests and publish alongside the container itself. Then a tool (such as Kubewarden) could gate running of the container based on the passing tests. + # How it Works [how-it-works]: #how-it-works -Add a field in the `layer.toml` spec called `additionalArtifact` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. +Add an optional file per layer called `artifacts.toml` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. -For the WASM usecase, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following `layer.toml`: +For the WASM use-case, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following `artifacts.toml`: ``` -[types] -publish = false -cache = false - -[[additionalArtifact]] +[[artifact]] tag = "wasm" -file = "mywasmprogram.wasm" # Should be a file in $LAYERS_DIR +file = "mywasmprogram.wasm" # Should be a file relative to the current $LAYERS_DIR -[[additionalArtifact.labels]] +[[artifact.labels]] ConfigMediaType = "application/vnd.wasm.config.v1+json" ContentLayerMediaType = "application/vnd.wasm.content.layer.v1+wasm" ``` +As a note: to keep this generic to any potential OCI artifact, it seems best to provide a labelling mechanism instead of prescribing certain fields. # Migration [migration]: #migration @@ -67,7 +66,7 @@ This functionality should be purely additive and not break existing buildpacks. # Drawbacks [drawbacks]: #drawbacks -This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. I think this can be mitigated with good docs and platform output. +This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. This can be mitigated with good docs and platform output. Adding a change to make the standard image output optional seems like a larger change than is appropriate at the moment. # Alternatives [alternatives]: #alternatives @@ -88,5 +87,5 @@ None that I'm aware of. # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes -The main addition to the spec is the addition of the fields in layer.toml as well as the changes to lifecycle to see them. +The only addition to the spec is the addition of the artifacts.toml as well as the changes to lifecycle to parse and act on the new config. From f40eafd396f4698e736b9443be221ee2f85242fe Mon Sep 17 00:00:00 2001 From: wanjunlei Date: Thu, 18 Aug 2022 14:15:50 +0800 Subject: [PATCH 154/372] RFC to support push image to isecure registries Signed-off-by: wanjunlei --- text/0000-support-insecure-registries.md | 72 ++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 text/0000-support-insecure-registries.md diff --git a/text/0000-support-insecure-registries.md b/text/0000-support-insecure-registries.md new file mode 100644 index 000000000..50d1d7aeb --- /dev/null +++ b/text/0000-support-insecure-registries.md @@ -0,0 +1,72 @@ +# Meta +[meta]: #meta +- Name: Support for pull images from or push images to insecure image registries. +- Start Date: 2022-08-17 +- Author(s): wanjunlei +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This RFC describes how to pull images form or push images to insecure image registries when build images using buildpacks. + +# Definitions +[definitions]: #definitions + +Insecure image registry - The insecure image registry mentioned in this RFC refers to the image registry using the http protocol, and with a non-internal ip or a domain name. Because currently buildpacks can access inecure image registries using internal ip. + +# Motivation +[motivation]: #motivation + +To fix issue [Support insecure registries in non-daemon case](https://github.com/buildpacks/lifecycle/issues/524). + +# What it is +[what-it-is]: #what-it-is + +With this RFC, user can use images in insecure image registries to build image, and push the compiled image to the insecure image registries like this. + +```shell +creator --run-image=develoment-registry.com/run-java:v16 --insecure-registry=develoment-registry.com --insecure-registry=testing-registry.com testing-registry.com/java-sample:latest +``` + +The flag `--insecure-registry` will add to analyzer, export, restor, rebaser and image tool too. + +# How it Works +[how-it-works]: #how-it-works + +With this [PR](https://github.com/buildpacks/imgutil/pull/154), the component that buildpacks used to operate images, already supports pulling images from or pushing images to insecure registries. +We should create a image with insecure registry by calling [NewImage](https://github.com/buildpacks/imgutil/blob/main/remote/remote.go#L119) like this + +```shell +remote.NewImage(imageName, authn.DefaultKeychain, withRegistrySetting("mydomain.registry.com:1080", true, false) +``` +> The second parameter specify whether the registry is insecure. +> The third parameter can be always false. + +It is necessary to judge whether the registry where this image is located is insecure. If so, it needs to set the insecure registry through `WithRegistrySetting`. + +The `NewImage` can set base image and prev image using `FromBaseImage` and `WithPreviousImage`, so it may need to call `WithRegistrySetting` multiple times to set up multiple insecure registries. + +# Drawbacks +[drawbacks]: #drawbacks + +N/A + +# Alternatives +[alternatives]: #alternatives + +N/A + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +A new flag `--insecure-registry` will add to analyzer, creator, export, restor, rebaser and image tool. \ No newline at end of file From 19d9dfdefc8e6e9f37cbcc2807f7f5a8699d594a Mon Sep 17 00:00:00 2001 From: Andrew Gracey Date: Thu, 25 Aug 2022 08:51:22 -0700 Subject: [PATCH 155/372] Accept PR suggestion -- removal of boilerplate Co-authored-by: Joe Kutner --- text/0000-additonal-oci-artifacts.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md index ccf1f6fb0..a4f3b7593 100644 --- a/text/0000-additonal-oci-artifacts.md +++ b/text/0000-additonal-oci-artifacts.md @@ -17,7 +17,6 @@ Allow buildpack authors to specify additional artifacts from their buildpacks to # Definitions [definitions]: #definitions -Make a list of the definitions that may be useful for those reviewing. Include phrases and words that buildpack authors or other interested parties may not be familiar with. - WebAssembly (WASM) -- Portable binaries to be run with a WebAssembly virtual machine. - [OCI Artifact](https://github.com/opencontainers/artifacts/blob/main/definitions-terms.md#media-type) -- Any blob of data that can be stored in an OCI registry. From the spec: From 6260996d4175c86e9410173b63d7f47e120c20ff Mon Sep 17 00:00:00 2001 From: wanjunlei Date: Fri, 26 Aug 2022 09:07:49 +0800 Subject: [PATCH 156/372] resolve conversation Signed-off-by: wanjunlei --- text/0000-support-insecure-registries.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/text/0000-support-insecure-registries.md b/text/0000-support-insecure-registries.md index 50d1d7aeb..dac574317 100644 --- a/text/0000-support-insecure-registries.md +++ b/text/0000-support-insecure-registries.md @@ -12,18 +12,20 @@ # Summary [summary]: #summary -This RFC describes how to pull images form or push images to insecure image registries when build images using buildpacks. +This RFC describes how to pull images from or push images to insecure image registries when building images using buildpacks. # Definitions [definitions]: #definitions -Insecure image registry - The insecure image registry mentioned in this RFC refers to the image registry using the http protocol, and with a non-internal ip or a domain name. Because currently buildpacks can access inecure image registries using internal ip. +Insecure image registry - The insecure image registry mentioned in this RFC refers to the image registry using the http protocol, and with a non-internal ip or a domain name. Because currently buildpacks can access insecure image registries using internal ip. # Motivation [motivation]: #motivation To fix issue [Support insecure registries in non-daemon case](https://github.com/buildpacks/lifecycle/issues/524). +To pull images from insecure image registries when building images using buildpacks, and push target images to insecure image registries after the build is complete. + # What it is [what-it-is]: #what-it-is From 43500616d9cf1709718140e8e42e95bb78b61a9a Mon Sep 17 00:00:00 2001 From: wanjunlei Date: Tue, 30 Aug 2022 09:25:28 +0800 Subject: [PATCH 157/372] resolve conversation Signed-off-by: wanjunlei --- text/0000-support-insecure-registries.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-support-insecure-registries.md b/text/0000-support-insecure-registries.md index dac574317..d691fe5dd 100644 --- a/text/0000-support-insecure-registries.md +++ b/text/0000-support-insecure-registries.md @@ -35,7 +35,7 @@ With this RFC, user can use images in insecure image registries to build image, creator --run-image=develoment-registry.com/run-java:v16 --insecure-registry=develoment-registry.com --insecure-registry=testing-registry.com testing-registry.com/java-sample:latest ``` -The flag `--insecure-registry` will add to analyzer, export, restor, rebaser and image tool too. +The flag `--insecure-registry` will be added to analyzer, export, restorer, rebaser and image tool too. # How it Works [how-it-works]: #how-it-works @@ -71,4 +71,4 @@ N/A # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes -A new flag `--insecure-registry` will add to analyzer, creator, export, restor, rebaser and image tool. \ No newline at end of file +A new flag `--insecure-registry` will add to analyzer, creator, export, restorer, rebaser and image tool. \ No newline at end of file From 234c22049c67d162639354ae100a21720032ae72 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Thu, 1 Sep 2022 07:38:36 +0100 Subject: [PATCH 158/372] Remove overrides concept End-users can provide arguments as key/value pairs. The key must match the name of a prompt in prompts.toml. --- text/0000-pack-buildpack-new-alternatives.md | 85 ++++++-------------- 1 file changed, 26 insertions(+), 59 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 17b3ee142..4a62a04d9 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -40,12 +40,13 @@ Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](ht The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell scaffolding from a cloned git repoistory. -The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--stacks` command line flags. These command line flags will be preserved in `pack buildpack create`. However, where a flag is omitted, the end-user will be prompted to provide an appropriate value. +The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--stacks` command line flags and the buildpack id as a positional argument. These command line flags and positional arguments will be replaced with user prompts `pack buildpack create`. A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```command -$ pack buildpack create example/bash +$ pack buildpack create +✔ Enter an ID for this buildpack: example/bash ✔ Enter a directory in which to scaffold the project: bash_buildpack Use the arrow keys to navigate: ↓ ↑ → ← ? Choose the buildpack API version (use the default if you are unsure): @@ -56,16 +57,15 @@ Use the arrow keys to navigate: ↓ ↑ → ← Created project in bash_buildpack ``` -The user can skip common prompts by providing `--api`, `--path` or `--stacks` as command line flags. Running the same example as above, but providing both `--api` and `--stacks`, but not `--path`, results in the user being prompted only to provide the output directory. We provide more detail on how flags override prompts in a subsection below. +The user can skip prompts by providing `--arg key=value` as command line flags. Each of the prompts from the previous interactive invocation can be skipped by providing appropriate pairs of key and value. ```command -$ pack buildpack create example/bash --api 0.8 --stacks io.buildpacks.samples.stacks.bionic -✔ Enter a directory in which to scaffold the project: bash_buildpack +$ pack buildpack create --arg ProjectDirectory=bash_buildpack --arg BuildpackApi=0.8 --arg BuildpackID=example/bash --arg BuildpakStacks=io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as follows, a full specification of `prompts.toml` is provided in the following subsection. +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding could be structured as follows, a full specification of `prompts.toml` is provided in the following subsection. ```bash . @@ -93,21 +93,7 @@ choices=["0.7", "0.8"] ... ``` -Answers to prompts can be provided in an `overrides.toml` file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. - -```command -$ pack buildpack create example/bash --override='ProjectDirectory="test_example"' --override='BuildpackApi=0.8' -✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic -Created project in bash_buildpack -``` - -The above overrides are equivalent to: - -```command -$ pack buildpack create example/bash --path test_example --api 0.8 -✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic -Created project in bash_buildpack -``` +A specification of `prompts.toml` is provided below. Multiple templates can be managed in a single repository, referred to as a **template colleciton**. @@ -143,11 +129,12 @@ Created project in go_buildpack A user may specify the choice of template from a template collection via a `--sub-path` flag. A user may choose a `bash` template from a template collection and provide the api, output directory and stacks as command line flags: ```command -$ pack buildpack create example/golang --template https://github.com/AidanDelaney/cnb-buildpack-templates \ +$ pack buildpack create --template https://github.com/AidanDelaney/cnb-buildpack-templates \ --sub-path Go \ - --api 0.8 \ - --path go_buildpack \ - --stacks io.buildpacks.samples.stacks.bionic + --arg BuildpackID=example/golang \ + --arg BuildpackAPI=0.8 \ + --arg ProjectDirectory=go_buildpack \ + --arg BuildpackStacks=io.buildpacks.samples.stacks.bionic Created project in go_buildpack ``` @@ -195,7 +182,7 @@ name="ProjectDirectory" prompt="Enter a directory" ``` -The following `prompts.toml` is invalid as the `default` and `choices` fields are mutually excusive: +The following `prompts.toml` is invalid as the `default` and `choices` fields are mutually exclusive: ```toml [[prompt]] @@ -205,44 +192,24 @@ default="/tmp" choices=["/tmp", "/home"] ``` -### Overrides +## Arguments -If a template contains both a `prompts.toml` file and an `overrides.toml` file, then the `overrides.toml` file provides values for variables introduced in `prompts.toml`. +Arguments may be provided as command line arguments. For example `pack buildpack create --arg ProjectDirectory=/tmp` provides a value, `/tmp`, for the variable identified by the name `ProjectDirectory`. -Given a `prompts.toml` file +When an argument, `key=value`, is provided then `pack buildpack create` must not prompt the user with the prompt identified by `name=key`. -```toml -[[prompt]] -name="ProjectDirectory" -prompt="Enter a directory in which to scaffold the project" -``` +Where an `--arg key=value` is provided and no prompt identified by `name=key` exists in `prompts.toml`, then the provided argument is ignored. -And an `overrides.toml` file +Choices constrain values for a given variable. Where a provided `value` does not match one of the `choices` then an error is returned. That is to say, given -```toml -ProjectDirectory="/tmp" +``` +[[prompt]] +name="BuildpackApi" +prompt="Choose the buildpack API version (use the default if you are unsure)" +choices=["0.7", "0.8"] ``` -Then the value "/tmp" is interpreted as the value of the variable identified by `ProjectDirectory`. - - -Overrides may also be provided as command line arguments. For example `pack buildpack create --overrides ProjectDirectory=/tmp` provides a value, `/tmp`, for the variable identified by the name `ProjectDirectory`. - -Overrides provided as command line flags have **higher precedence** than overrides provided in `overrides.toml`. - -When an override is provided `pack buildpack create` must not prompt the user for overriden values. - -## `pack buildpack create` flags as prompts variables - -If specified, the following `pack buildpack create` flags override the value of variables with the listed identifier: - -| command line flag | variable name | -|-------------------|--------------------| -| `--path` | `ProjectDirectory` | -| `--api` | `BuildpackApi` | -| `--stacks` | `BuildpackStacks` | - -In addition, the required positional argument to `pack buildpack create`, eg: `pack buildpack create example/bash`, overrides the variable identified by `BuildpackID`. +and given a CLI invocation `pack buildpack create --arg BuildpackApi=0.6` the choices are restricted to `["0.7", "0.8"]`. Therefore, `0.6` is an invalid argument and an error is returned to the end-user. ## `pack buildpack create` Substitutions @@ -250,6 +217,8 @@ The operation of variable substitution follows the operation of Go [`text/templa Variables may be used in files, for example where a `prompts.toml` defines a variable with identifier `ProjectDirectory` then the expression `{{.ProjectDirectory}}` in files is replaced with the value of the variable identified by `ProjectDirectory`. In addition, the expression `{{.ProjectDirectory}}` may be used in template file paths. The generated project substitutes file paths with variable expressions with the value of the variable. +When a source file contains a Go `text/template` style expression and the variable name does not appear in `prompts.toml`, then the `text/template` style expression is not replaced. For example, if a `README.md` file contains the expression `{{.Example}}` and `Example` is not a variable defined in `prompts.toml`, then the string `{{.Example}}` is not replaced in `README.md`. + # Migration [migration]: #migration @@ -297,8 +266,6 @@ There are many, many competing implementations of project scaffolding tools. # Unresolved Questions [unresolved-questions]: #unresolved-questions -* Where do the template repositories live for `bash` and `golang`? - * * What team maintains template repositories? * Buildpack Author Tooling team From 83d8f3a6810e69a2c20314a58fbf980b03c55752 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Fri, 2 Sep 2022 13:40:29 +0100 Subject: [PATCH 159/372] Apply suggestions from code review Co-authored-by: Natalie Arellano Co-authored-by: Joe Kutner Co-authored-by: Javier Romero --- text/0000-rfc-process.md | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index 8d6516305..9e2b0d695 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -12,7 +12,7 @@ # Summary [summary]: #summary -The "RFC" (request for comments) process provide a consistent and controlled path for new features and other substantive changes to enter the project. +The "RFC" (request for comments) process provides a consistent and controlled path for new features and other substantive changes to enter the project. The RFC process: * provides a set of checkpoints that ensure changes align with the overall technical and strategic vision for the process. @@ -57,7 +57,7 @@ The RFC process **is not**: **Voting Window**: The time period between a call for votes and the voting end date is referred to as the voting window. -**Lazy Consensus**: Voting on project level RFCs is done by lazy consensus. Any project member with a binding vote who has not voted by the end date is assumed to assent to the RFC. +**Lazy Consensus**: Voting on RFCs is done by lazy consensus. Any project member with a binding vote who has not voted by the end date is assumed to assent to the RFC. **Binding Vote**: Binding Votes have formal power within the RFC process. A single no vote by a project member with a binding vote is sufficient to reject an RFC. @@ -80,7 +80,7 @@ As the project grows our process needs to grow with it. Compared to the previous ### What Types of Changes Require an RFC? -Any "substantive" change to the project requires an RFC. substantive includes but is not limited to: +Any "substantive" change to the project should require an RFC. substantive includes but is not limited to: * changes to the specification. * the adoption, creation, or deprecation of a component (e.g. a new platform implementation, a new shared library, a new system buildpack). * new features (e.g. a new pack command, a new flag on an existing pack command, an addition to the buildpack API) @@ -88,7 +88,7 @@ Any "substantive" change to the project requires an RFC. substantive includes bu * any major re-architecture especially if it has noteworthy implications for security or performance. * introduction of new processes or changes to our existing processes including the RFC process. -If there is any doubt, maintainers should prefer opening an "unecessary" RFC to surprising users was unexpectedly impactful changes. +If there is any doubt, maintainers should prefer opening an "unnecessary" RFC over surprising users with unexpectedly impactful changes. #### Project vs Team RFCs @@ -128,7 +128,7 @@ In many but not all cases the appropriate team to own an RFC will be obvious. Wh * Which team is responsible for the components most impacted by the proposed change? * Which team is enthusiastic about supporting the change? -If a home cannot be found for a draft RFC it remains in draft until one can be found. This does not necessary mean that the RFC is not a good idea or isn't something the project will take up eventually. It may simply be that the project does not have the bandwidth to prioritize this particular change at this particular time. +If a home cannot be found for a draft RFC it remains in draft until one can be found. This does not necessarily mean that the RFC is not a good idea or isn't something the project will take up eventually. It may simply be that the project does not have the bandwidth to prioritize this particular change at this particular time. #### Stewardship @@ -149,7 +149,7 @@ The RFC may not be edited during the voting window. For project RFCs, all TOC members and all team leads are given a binding vote. -For team RFCs, all team maintainers are given a binding vote. +Additionally, for team RFCs, all team maintainers are given a binding vote. Votes are cast via reviews on the RFC PR. Accepting the PR signifies a yes vote while a request for changes signifies a no vote. If all members with a binding vote vote in the affirmative, the voting window may close early. @@ -159,7 +159,7 @@ If the end date of the vote arrives without a no vote from a member with binding ##### Rejection -If a single no vote is cast before the arrival of the end date the RFC is immediately rejected and the PR should be closed. +If a single no vote from a member with binding vote is cast before the arrival of the end date the RFC is immediately rejected and the PR should be closed. The same RFC may be re-opened and brought to a vote again in the future assuming the concerns that lead to the no vote are addressed. @@ -197,7 +197,7 @@ The RFC process should be amended through the RFC process. However, the TOC rese | `status/ready-for-review` | The Steward applies this to an RFC when they deem it ready for review by a broader audience. | `status/voting` | The Steward applies this label when the voting window opens | `status/accepted` | The Steward applies this label to accepted RFCs. -| `status/rejected` | The Steard applies this label to a rejected RFCs. +| `status/rejected` | The Steward applies this label to rejected RFCs. # Migration [migration]: #migration @@ -207,30 +207,25 @@ This RFC should be accepted via the existing RFC process. The new process will t # Drawbacks [drawbacks]: #drawbacks -Why should we *not* do this? - -TODO +None # Alternatives [alternatives]: #alternatives -- What other designs have been considered? -- Why is this proposal the best? -- What is the impact of not doing this? - -TODO +None # Prior Art [prior-art]: #prior-art TODO -* Borrows heavily from TEP and KEP processes +* [Tekton Enhancement Proposal Process (TEP)](https://github.com/tektoncd/community/blob/main/teps/0001-tekton-enhancement-proposal-process.md) +* [Kubernetes Enhancement Proposals (KEP)](https://github.com/kubernetes/enhancements/blob/master/keps/README.md) # Unresolved Questions [unresolved-questions]: #unresolved-questions -TODO +None # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From e932fa273042bbf848c4e3cad7984988857f0ec0 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 6 Sep 2022 05:52:05 +0100 Subject: [PATCH 160/372] Drop prompt locale Incorporating the locale introduces too much complexity in the initial RFC. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 4a62a04d9..201c3650c 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -148,13 +148,10 @@ The prompts are contained in a TOML file. The `prompts.toml` file contains an a |---------------|----------|------------------------------------------------------------------------------------------------------------------------------------------|----------------| | name | Yes | The variable identifier. Must be unique within the `prompts.toml` file. | string | | prompt | Yes | The default prompt question to ask the user | string | -| prompt.locale | No | A translation of the default prompt into the locale specified. | | | required | No | Evaluates to `false` if not provided. Specifies whether the user **must** provide a value for the variable identified by `name`. | boolean | | default | No | A default value for the variable identified by `name`. Mutually exclusive to `choices` | string | | choices | No | A list of default values from which the user may choose a value for the variable identified by `name`. Mutually exclusive to `default`. | list of stings | -Prompt locales are specified following [IETF BCP 47](https://tools.ietf.org/rfc/bcp/bcp47.txt). As such `prompt.en-US` is the translation of `prompt` into US English and `prompt.es-MX` is the translation of `prompt` into Mexican Spanish. - For example, the following is a valid `prompts.toml` file: ```toml From 364482e9d10cad4299ae5c7d516e624d6192e721 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 15 Sep 2022 09:33:48 -0500 Subject: [PATCH 161/372] Update text/0000-rfc-process.md Co-authored-by: Aidan Delaney Signed-off-by: Joe Kutner --- text/0000-rfc-process.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index 9e2b0d695..d9e994fc0 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -217,7 +217,6 @@ None # Prior Art [prior-art]: #prior-art -TODO * [Tekton Enhancement Proposal Process (TEP)](https://github.com/tektoncd/community/blob/main/teps/0001-tekton-enhancement-proposal-process.md) * [Kubernetes Enhancement Proposals (KEP)](https://github.com/kubernetes/enhancements/blob/master/keps/README.md) From 5d8ba9905f135878112d8617e33d27092c2bf5e1 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 15 Sep 2022 09:34:05 -0500 Subject: [PATCH 162/372] Update text/0000-rfc-process.md Co-authored-by: Aidan Delaney Signed-off-by: Joe Kutner --- text/0000-rfc-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index d9e994fc0..cfb55ba28 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -80,7 +80,7 @@ As the project grows our process needs to grow with it. Compared to the previous ### What Types of Changes Require an RFC? -Any "substantive" change to the project should require an RFC. substantive includes but is not limited to: +Any "substantive" change to the project should require an RFC. Substantive includes but is not limited to: * changes to the specification. * the adoption, creation, or deprecation of a component (e.g. a new platform implementation, a new shared library, a new system buildpack). * new features (e.g. a new pack command, a new flag on an existing pack command, an addition to the buildpack API) From 24087867121b7eded84b174da335bb21ceebaf72 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 28 Sep 2022 10:05:12 -0500 Subject: [PATCH 163/372] included review comments from #218 Signed-off-by: Joe Kutner --- text/0000-rfc-process.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/text/0000-rfc-process.md b/text/0000-rfc-process.md index cfb55ba28..7e30b943e 100644 --- a/text/0000-rfc-process.md +++ b/text/0000-rfc-process.md @@ -51,7 +51,7 @@ The RFC process **is not**: **Steward**: The steward of an RFC is responsible for shepherding an RFC through the process, including working with the author to ensure RFC completeness and quality, and building consensus among stakeholders. -**Call for Votes**: When an RFC is deemed ready by a team lead, that team lead initiates the voting process with a call for votes. At this point the RFC is closed to modification. +**Call for Votes**: When an RFC is deemed ready by its steward, the steward initiates the voting process with a call for votes. At this point the RFC is closed to modification. **End Date**: When a call for votes is initiated, an end date for voting is set. Any person wishing to vote on an RFC must do so by the end date. @@ -82,7 +82,7 @@ As the project grows our process needs to grow with it. Compared to the previous Any "substantive" change to the project should require an RFC. Substantive includes but is not limited to: * changes to the specification. -* the adoption, creation, or deprecation of a component (e.g. a new platform implementation, a new shared library, a new system buildpack). +* the adoption, creation, or deprecation of a component (e.g. a new platform implementation, a new shared library, a new system buildpack). * new features (e.g. a new pack command, a new flag on an existing pack command, an addition to the buildpack API) * any major refactor that affects consumers of our libraries or materially impacts contribution. * any major re-architecture especially if it has noteworthy implications for security or performance. @@ -112,7 +112,7 @@ Given the nature of our project many RFCs will happen at the project level. Howe If you believe an RFC should be written but are either not prepared to author it personally or not prepared to do it _now_, please open an issue on this repo with a high level description of the desired change. This helps us keep track of good ideas and match make ambitious contributors with appropriately sized challenges. #### Drafting an RFC -All RFCs begin life as a draft. Anyone wishing to propose a change to the project should create a draft RFC by: +All RFCs begin life as a pull request. Anyone wishing to propose a change to the project should create an RFC pull request by: - Fork the RFC repo: - Copy `0000-template.md` to `text/0000-my-feature.md` (where 'my-feature' is descriptive. don't assign an RFC number yet). @@ -143,7 +143,7 @@ The steward and their team should: Ideally the voting should be a formality and not a moment to discover new disagreement, consensus already haven been driven by the steward. -When the steward deems the RFC ready and likely to be accepted they should formally call for votes and set an end date for voting. This process does not prescribe a length for the voting window, but stewards should make a good faith effort to ensure that all interested parties, and in particular those with binding votes, have adequate opportunity to review the finalized RFC and cast votes. +When the steward deems the RFC ready and likely to be accepted they should formally call for votes and set an end date for voting. Stewards should make a good faith effort to ensure that all interested parties, and in particular those with binding votes, have adequate opportunity to review the finalized RFC and cast votes. A minimum time period of a week (unless all of the binding votes are cast) is a good guideline. The RFC may not be edited during the voting window. @@ -151,7 +151,7 @@ For project RFCs, all TOC members and all team leads are given a binding vote. Additionally, for team RFCs, all team maintainers are given a binding vote. -Votes are cast via reviews on the RFC PR. Accepting the PR signifies a yes vote while a request for changes signifies a no vote. If all members with a binding vote vote in the affirmative, the voting window may close early. +Votes are cast via reviews on the RFC PR. Accepting the PR signifies a yes vote while a request for changes signifies a no vote. If all members with a binding vote vote in the affirmative, the voting window may close early. In order for a RFC to be approved, at least 1 yes vote. In the rare case there are no votes, the Calling for Votes process may be followed again or the RFC may be closed. ##### Acceptance @@ -165,6 +165,8 @@ The same RFC may be re-opened and brought to a vote again in the future assuming ##### Implementation +After the RFC has been accepted, any team or individual may begin implementing the changes it defines. + ##### Amending an RFC While we should strive to get the details right the first time, by doing our due diligence including a proof of concept implementation for larger/riskier RFCs, there will be times when we discover during the process of implementation that something about the original plan was incomplete or needs adjustment. Small changes may be PR'ed by the implementer and merged upon approval by the steward. It is the responsibility of the Steward to determine what changes are minor enough to qualify as an amendment and which are fundamental and/or controversial enough to require a new superseding RFC. @@ -177,7 +179,7 @@ PRs that amend an RFC should add an amendment to the [History](../0000-template. ### Amending the RFC Process -The RFC process should be amended through the RFC process. However, the TOC reserves the right to change the process via a super-majority vote in the unlikely even that the process prove so irreparably flawed as to preclude its amendment via the process. +The RFC process should be amended through the RFC process. However, the TOC reserves the right to change the process via a super-majority vote in the unlikely even that the process proves so irreparably flawed as to preclude its amendment via the process. # How it Works [how-it-works]: #how-it-works @@ -196,9 +198,12 @@ The RFC process should be amended through the RFC process. However, the TOC rese | `status/needs-steward` | Applied to all new RFCs. Removed once the RFC is accepted by a steward or closed. | `status/ready-for-review` | The Steward applies this to an RFC when they deem it ready for review by a broader audience. | `status/voting` | The Steward applies this label when the voting window opens +| `status/steward-assigned` | The Steward applies this label when adopting the RFC along with self-assigning the PR | `status/accepted` | The Steward applies this label to accepted RFCs. | `status/rejected` | The Steward applies this label to rejected RFCs. +The `Final Comment Period` label will no longer be used. + # Migration [migration]: #migration @@ -207,7 +212,7 @@ This RFC should be accepted via the existing RFC process. The new process will t # Drawbacks [drawbacks]: #drawbacks -None +* Lazy consensus favors velocity and moving things forward, over rigor and thoroughness. This could result in a higher volume of changes that have some unforeseen consequences. This is a tradeoff we are consciously making. # Alternatives [alternatives]: #alternatives From b1eeddf1da0105203e1d60ec0303b1932e2950f6 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 30 Sep 2022 18:16:19 -0500 Subject: [PATCH 164/372] WIP - initial version Signed-off-by: Juan Bustamante --- ...00-governance-component-maintainer-role.md | 124 ++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 text/0000-governance-component-maintainer-role.md diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md new file mode 100644 index 000000000..06d89fe8a --- /dev/null +++ b/text/0000-governance-component-maintainer-role.md @@ -0,0 +1,124 @@ +# Meta +[meta]: #meta +- Name: add component maintainer role +- Start Date: 2022-09-39 +- Author(s): @jjbustamante +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +This RFC proposes the creation of a new role **component maintainer** in the project governance responsible of executing maintainer duties on specific repositories in a component of each team. + +# Definitions +[definitions]: #definitions + +- teams: Group of individuals focused on narrower sets of concerns related to specific aspects of the project. Example: implementation team, platform team. +- team lead: +- maintainers: Individuals that maintain specific code repositories. +- contributor: +- software component: is a software unit with a well-defined interface and explicitly specified dependencies. +- component maintainer: the proposed role in this RFC. + +# Motivation +[motivation]: #motivation + +- Why should we do this? + +Our current governance process defines [teams](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#teams) and each team is responsible for maintaining some number of software components, these teams are organized internally with 3 different roles: + - Team Leads + - Maintainers + - Contributors + +The process does not take into consideration the sizes of the software components that a maintainer must take accountability for, as the number and size of all the software components in a team increases we may need to distribute the responsibilities in a different way. + +- What use cases does it support? + +It provides a mechanism to handle the increases of complexity of the source code of each component maintain by a team, when a team lead or maintainer determines a software component is big enough to be delegated to a contributor that desires to specialize and has the know-how to be responsible of a component they can nominate him/her to become **component maintainer** + +- What is the expected outcome? + +# What it is +[what-it-is]: #what-it-is + +For each repository of the component under his responsibility the **component maintainer** are: + +- reviewing, approving, and merging pull requests. +- planning release milestones, and releasing components versions +- edit, label, assign issues + +# How it Works +[how-it-works]: #how-it-works + +## Guidelines nominate component maintainer + +A Team Lead or a Maintainer will follow this guideline to nominate a component maintainer for some software component inside their team + +- A software component developed outside CNB project was [accepted](https://github.com/buildpacks/community/blob/main/contributors/guide.md#component) and current team do not have the know-how or experience to handle it. + + +# Migration +[migration]: #migration + +None + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? +- Yet another role +- We decide to wait until the problem arises + +# Alternatives +[alternatives]: #alternatives + +## Do Nothing + +If no new role is created, `maintainer` will continue to be accountable and responsible of all the software components inside a team. + + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + + + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +None + +# History +[history]: #history + +None + + From ccc6951302c0ca692f83fcf69e647de874880985 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 4 Oct 2022 11:46:01 -0500 Subject: [PATCH 165/372] adding diagrams and more explanations Signed-off-by: Juan Bustamante --- ...00-governance-component-maintainer-role.md | 30 ++++++++++++++----- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index 06d89fe8a..d7776c136 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -12,15 +12,15 @@ # Summary [summary]: #summary -This RFC proposes the creation of a new role **component maintainer** in the project governance responsible of executing maintainer duties on specific repositories in a component of each team. +This RFC proposes the creation of a new role **component maintainer** in the project governance, responsible of executing maintainer duties on the repositories in a specific component under a CNB team. # Definitions [definitions]: #definitions - teams: Group of individuals focused on narrower sets of concerns related to specific aspects of the project. Example: implementation team, platform team. -- team lead: -- maintainers: Individuals that maintain specific code repositories. -- contributor: +- maintainers: individual that maintain specific code repositories. +- team lead: is a maintainer who has special responsibilities for representing team concerns at the project level +- contributor: individual who make regular contributions to a team (documentation, code reviews, responding to issues, participation in proposal discussions, contributing code, etc.) - software component: is a software unit with a well-defined interface and explicitly specified dependencies. - component maintainer: the proposed role in this RFC. @@ -34,18 +34,32 @@ Our current governance process defines [teams](https://github.com/buildpacks/com - Maintainers - Contributors -The process does not take into consideration the sizes of the software components that a maintainer must take accountability for, as the number and size of all the software components in a team increases we may need to distribute the responsibilities in a different way. +The process does not take into consideration the sizes of the software components that a maintainer must take accountability for or the expectations of the community contributors who made want to specialize in certain pieces of the projects, as the number and size of all the software components in a team increases we may need to distribute the responsibilities in a different way. - What use cases does it support? -It provides a mechanism to handle the increases of complexity of the source code of each component maintain by a team, when a team lead or maintainer determines a software component is big enough to be delegated to a contributor that desires to specialize and has the know-how to be responsible of a component they can nominate him/her to become **component maintainer** +It provides a mechanism to handle the increases of complexity of the source code of each component maintain by a team, when a team lead or maintainer determines a software component is big enough to be delegated to a contributor that desires to specialize and has the know-how to be responsible of a component they can nominate him/her to become **component maintainer** and delegates some of the day to day activities to this person. - What is the expected outcome? +A new role and guideline on how to nominate individuals to this role will be include it in our governance process. + # What it is [what-it-is]: #what-it-is -For each repository of the component under his responsibility the **component maintainer** are: +CNB maintainer contributor responsibility can be describe as: `is in charge of the day to day maintenance of the team's projects` + +![](https://i.imgur.com/yXsLK6N.png) + +As we can see in diagram above, a CNB team takes care of `N` number of [software components](#definitions) and the project contributors from the community make them contributions to any of those components. + +Depending on the team, these components can increase in size or complexity, or there could be someone from the community that wants to specialize their contributions on certain components without taking the responsibility of become a **team maintainer**. + +The proposal is to incorporate a **component maintainer** role. + +![](https://i.imgur.com/rWElkCw.png) + +The **component maintainer** will take under his/her responsibility a well defined software component in a CNB team and for each repository will be allow to: - reviewing, approving, and merging pull requests. - planning release milestones, and releasing components versions @@ -59,7 +73,7 @@ For each repository of the component under his responsibility the **component ma A Team Lead or a Maintainer will follow this guideline to nominate a component maintainer for some software component inside their team - A software component developed outside CNB project was [accepted](https://github.com/buildpacks/community/blob/main/contributors/guide.md#component) and current team do not have the know-how or experience to handle it. - + # Migration [migration]: #migration From 4f335480cebcce1cc17c595580ee0a688fddfac2 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 7 Oct 2022 13:50:17 -0400 Subject: [PATCH 166/372] Query template arguments Provide `pack buildpack create --help` to query the arguments that exist in a template. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 29 ++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 201c3650c..b82fbdca1 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -208,6 +208,35 @@ choices=["0.7", "0.8"] and given a CLI invocation `pack buildpack create --arg BuildpackApi=0.6` the choices are restricted to `["0.7", "0.8"]`. Therefore, `0.6` is an invalid argument and an error is returned to the end-user. +Arguments for buildpack creation need to be discoverable. We intend to support this via the `--help` flag to `pack buildpack create`. Executing + +``` +pack buildpack create --help + +pack buildpack create +arguments offered by template + ProjectName (default: pyexample) + BuildpackApi 0.7, 0.8 (default 0.7) +``` + +shows a default help message. Invoking `--help` with a template url queries the prompts provided by the template: + +``` +pack buildpack create --help --template= +arguments offered by template + ProjectName (default: pyexample) + PythonVersion=python3.10, python3.9, python3.8 (default: python3.10) + NumDigits (default: 3) +``` +The possible values of a template collection can similarly be queried: + +``` +pack buildpack create --help --template= +templates available in collection + Go + bash +``` + ## `pack buildpack create` Substitutions The operation of variable substitution follows the operation of Go [`text/template`](https://pkg.go.dev/text/template). Prompts defined in `prompts.toml` are interpreted, the user may be prompted and set of variables is generated. The identifiers of variables are replaced with the value of the variable. From e0147b7949e777e515893f0763f068c0fea4eada Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 7 Oct 2022 14:53:16 -0500 Subject: [PATCH 167/372] Adding feedback from Navdeep Signed-off-by: Juan Bustamante --- ...00-governance-component-maintainer-role.md | 71 ++++++++++++++++--- 1 file changed, 61 insertions(+), 10 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index d7776c136..d0cecc58a 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -17,12 +17,12 @@ This RFC proposes the creation of a new role **component maintainer** in the pro # Definitions [definitions]: #definitions -- teams: Group of individuals focused on narrower sets of concerns related to specific aspects of the project. Example: implementation team, platform team. -- maintainers: individual that maintain specific code repositories. -- team lead: is a maintainer who has special responsibilities for representing team concerns at the project level -- contributor: individual who make regular contributions to a team (documentation, code reviews, responding to issues, participation in proposal discussions, contributing code, etc.) -- software component: is a software unit with a well-defined interface and explicitly specified dependencies. -- component maintainer: the proposed role in this RFC. +- **maintainers**: individual that maintain specific code repositories. +- **contributor**: individual who make regular contributions to a team (documentation, code reviews, responding to issues, participation in proposal discussions, contributing code, etc.) +- **teams**: Group of individuals focused on narrower sets of concerns related to specific aspects of the project. Example: implementation team, platform team. +- **team lead**: is a maintainer who has special responsibilities for representing team concerns at the project level +- **component maintainer**: the proposed role in this RFC. Individual that maintain specific code repositories of a software-component inside a CNB team. +- **software component**: is a software unit with a well-defined interface and explicitly specified dependencies. # Motivation [motivation]: #motivation @@ -36,6 +36,10 @@ Our current governance process defines [teams](https://github.com/buildpacks/com The process does not take into consideration the sizes of the software components that a maintainer must take accountability for or the expectations of the community contributors who made want to specialize in certain pieces of the projects, as the number and size of all the software components in a team increases we may need to distribute the responsibilities in a different way. +Compared to the model based on 3 roles, this new role should: +- **re-balance responsibilities inside a team** empower contributors to take ownership of key components align with their technical skills or career path expectations +- **support maintainers on their role's goals** one responsibility of a maintainer is `growing the team by mentoring aspiring contributors and maintainers`, this new role offers a growing path for some contributors what would like to become maintainers. + - What use cases does it support? It provides a mechanism to handle the increases of complexity of the source code of each component maintain by a team, when a team lead or maintainer determines a software component is big enough to be delegated to a contributor that desires to specialize and has the know-how to be responsible of a component they can nominate him/her to become **component maintainer** and delegates some of the day to day activities to this person. @@ -55,9 +59,47 @@ As we can see in diagram above, a CNB team takes care of `N` number of [software Depending on the team, these components can increase in size or complexity, or there could be someone from the community that wants to specialize their contributions on certain components without taking the responsibility of become a **team maintainer**. -The proposal is to incorporate a **component maintainer** role. +### Example -![](https://i.imgur.com/rWElkCw.png) +Let's take for example the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team), which right now have 2 maintainers, and let's use the LOC (lines of code) metrics, for each of the components maintains by this team, to dimension the size of it. + +| Component | LOC | +|--------------------------|--------| +| [Pack]() | +58000 | +| [Tekton Tasks + Pipelines]() | +2150 | +| [CircleCI Pack Orb]() | +400 | + +**Note**: [Tokei](https://github.com/XAMPPRocky/tokei) tool was used to calculate the LOC of the repositories. + +#### Integration with the Cloud Native Ecosystem + +As part of the [CNB roadmap](https://github.com/buildpacks/community/blob/main/ROADMAP.md#integration-with-the-cloud-native-ecosystem) a `better out-of-the box Kubernetes and Docker integration` is a goal of the project and in order to do that, the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team) will have to include another software component that accomplish this goal. + +In case of this scenario, then an hypothetical update of the component table will be: + +| Component | LOC | +|--------------------------|--------| +| [Pack]() | +58000 | +| [Tekton Tasks + Pipelines]() | +2150 | +| [CircleCI Pack Orb]() | +400 | +| Kubernetes Native Platform implementation | ? | + +Because, this is hypothetical scenario, we actually don't have the size but, we can use [kpack]() as reference, because it is a Kubernetes Platform implementation of the CNB specification. Let's update the table again. + +| Component | LOC | +|--------------------------|--------| +| [Pack]() | +58000 | +| [Tekton Tasks + Pipelines]() | +2150 | +| [CircleCI Pack Orb]() | +400 | +| [Kpack]() | +57000 | + +As we can see, a new implementation of the [Platform Interface Specification](https://github.com/buildpacks/spec/blob/main/platform.md) could be as big as [pack]() but most important: +- It requires a specific knowledge in Kubernetes and everything related + +# How it Works +[how-it-works]: #how-it-works + +The proposal is to incorporate a **component maintainer** role. The **component maintainer** will take under his/her responsibility a well defined software component in a CNB team and for each repository will be allow to: @@ -65,8 +107,17 @@ The **component maintainer** will take under his/her responsibility a well defin - planning release milestones, and releasing components versions - edit, label, assign issues -# How it Works -[how-it-works]: #how-it-works +An updated version of the previous diagram shows graphically this new roles + +![](https://i.imgur.com/rWElkCw.png) + +### Example + +Let's come back to our previous example. + +#### Integration with the Cloud Native Ecosystem + +In this case, the existence of the **component maintainer** will provide the rules to the **platform team lead** or **platform maintainers** to nominate (following the guidelines describe in the next section) a **component maintainer** for [kpack]() when they consider is necessary. ## Guidelines nominate component maintainer From 2f57f3c3bec6e3a94f022407773f66982deb0445 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 10 Oct 2022 15:06:25 -0500 Subject: [PATCH 168/372] Adding alternatives section Signed-off-by: Juan Bustamante --- text/0000-governance-component-maintainer-role.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index d0cecc58a..cefad3ef7 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -7,7 +7,7 @@ - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) +- Supersedes: [#228](https://github.com/buildpacks/rfcs/pull/228) # Summary [summary]: #summary @@ -141,10 +141,15 @@ Why should we *not* do this? # Alternatives [alternatives]: #alternatives -## Do Nothing +### Do Nothing If no new role is created, `maintainer` will continue to be accountable and responsible of all the software components inside a team. +### Repository level ownership RFC + +The first attempt to addressed the problem was proposing the **repository level ownership** [RFC](https://github.com/buildpacks/rfcs/pull/228) but abandoned this idea because: +- A new role is more consistent with current governance structure +- A new role will provide more visibility to the community to identify individuals responsible for each repository. # Prior Art [prior-art]: #prior-art From 1bec239019e8e3bf5d7cec5bc9a9d78bbfad2305 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 10 Oct 2022 15:54:36 -0500 Subject: [PATCH 169/372] adding more detail to the guidelines Signed-off-by: Juan Bustamante --- text/0000-governance-component-maintainer-role.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index cefad3ef7..15b5e2e13 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -121,10 +121,14 @@ In this case, the existence of the **component maintainer** will provide the rul ## Guidelines nominate component maintainer -A Team Lead or a Maintainer will follow this guideline to nominate a component maintainer for some software component inside their team - -- A software component developed outside CNB project was [accepted](https://github.com/buildpacks/community/blob/main/contributors/guide.md#component) and current team do not have the know-how or experience to handle it. - +Follow this guideline to nominate a **component maintainer** for a software component inside their team + +- The software component must be defined under the [GOVERNANCE](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md) team section, for example: Platform Team -> kpack +- New **component maintainers** must already be contributors of the team +- New **component maintainers** must be nominate by a **team lead** or a **maintainer** of the team under the following scenarios: + - A software component developed outside CNB project was [accepted](https://github.com/buildpacks/community/blob/main/contributors/guide.md#component) under their team and current team do not have the know-how or experience to handle it. + - A community **contributor** have explicitly manifest the desire to become a **component maintainer** and the **team lead** or **maintainer** consider he/she has the skills and knowledge to take the responsibility and accountability of the component. +- New **component maintainers** must be elected by [super-majority](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#supermajority) of the team’s maintainers # Migration [migration]: #migration From af2264835cd3c365245afbee3914286f63140df4 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 11 Oct 2022 09:47:30 -0500 Subject: [PATCH 170/372] Some fixes Signed-off-by: Juan Bustamante --- text/0000-governance-component-maintainer-role.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index 15b5e2e13..563360072 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -27,7 +27,7 @@ This RFC proposes the creation of a new role **component maintainer** in the pro # Motivation [motivation]: #motivation -- Why should we do this? +### Why should we do this? Our current governance process defines [teams](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#teams) and each team is responsible for maintaining some number of software components, these teams are organized internally with 3 different roles: - Team Leads @@ -40,11 +40,11 @@ Compared to the model based on 3 roles, this new role should: - **re-balance responsibilities inside a team** empower contributors to take ownership of key components align with their technical skills or career path expectations - **support maintainers on their role's goals** one responsibility of a maintainer is `growing the team by mentoring aspiring contributors and maintainers`, this new role offers a growing path for some contributors what would like to become maintainers. -- What use cases does it support? +### What use cases does it support? It provides a mechanism to handle the increases of complexity of the source code of each component maintain by a team, when a team lead or maintainer determines a software component is big enough to be delegated to a contributor that desires to specialize and has the know-how to be responsible of a component they can nominate him/her to become **component maintainer** and delegates some of the day to day activities to this person. -- What is the expected outcome? +### What is the expected outcome? A new role and guideline on how to nominate individuals to this role will be include it in our governance process. @@ -94,7 +94,7 @@ Because, this is hypothetical scenario, we actually don't have the size but, we | [Kpack]() | +57000 | As we can see, a new implementation of the [Platform Interface Specification](https://github.com/buildpacks/spec/blob/main/platform.md) could be as big as [pack]() but most important: -- It requires a specific knowledge in Kubernetes and everything related +- It requires a specific knowledge in [Kubernetes](https://kubernetes.io/) # How it Works [how-it-works]: #how-it-works @@ -113,15 +113,15 @@ An updated version of the previous diagram shows graphically this new roles ### Example -Let's come back to our previous example. +Let's come back to our previous [kpack](https://github.com/pivotal/kpack) example. #### Integration with the Cloud Native Ecosystem -In this case, the existence of the **component maintainer** will provide the rules to the **platform team lead** or **platform maintainers** to nominate (following the guidelines describe in the next section) a **component maintainer** for [kpack]() when they consider is necessary. +In this case, the existence of the **component maintainer** role will provide the rules to the **platform team lead** or **platform maintainers** to nominate (following the guidelines describe in the next section) a **component maintainer** for [kpack](https://github.com/pivotal/kpack) when they consider is necessary. ## Guidelines nominate component maintainer -Follow this guideline to nominate a **component maintainer** for a software component inside their team +Follow this guideline to nominate a **component maintainer** for a software component inside a team - The software component must be defined under the [GOVERNANCE](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md) team section, for example: Platform Team -> kpack - New **component maintainers** must already be contributors of the team From 04db1526da5aa0dc03c467e8444de957ba1ab400 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 21 Jun 2022 11:53:28 -0500 Subject: [PATCH 171/372] Initial version of the proposal to donate kpack to CNB project Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 313 +++++++++++++++++++++++++++++ 1 file changed, 313 insertions(+) create mode 100644 text/0000-kpack-donation-to-cnb.md diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md new file mode 100644 index 000000000..2fd41ec7f --- /dev/null +++ b/text/0000-kpack-donation-to-cnb.md @@ -0,0 +1,313 @@ +# Meta + +- Name: kpack donation to CNB +- Start Date: 2022-06-21 +- Author(s): [Juan Bustamante](https://github.com/jjbustamante/) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary + +This RFC proposes the donation of the open-source project [kpack](https://github.com/pivotal/kpack/) into the [Cloud Native Buildpack](https://buildpacks.io/) (CNB) ecosystem. + +# Definitions + +- [Kubernetes](https://kubernetes.io/) is an open-source system for automating deployment, scaling, and management of containerized applications. +- [Kpack](https://github.com/pivotal/kpack/) is a VMware-led open-source project that utilizes [Kubernetes](https://kubernetes.io/) primitives to build OCI images as a [platform](https://buildpacks.io/docs/concepts/components/platform/) implementation of [Cloud Native Buildpacks](https://buildpacks.io/). +- A Kubernetes native application is an application designed to run on Kubernetes platforms, managed by Kubernetes APIs and `kubectl` tooling and cohesively deployed on Kubernetes as a single object. + +# Motivation + +### Why should we do this? + +It will benefit the [CNB](https://buildpacks.io/) project by adding a tool to support an out-of-the box [Kubernetes](https://kubernetes.io/) integration, which is part of the [CNB](https://buildpacks.io/) [roadmap](https://github.com/buildpacks/community/blob/main/ROADMAP.md#integration-with-the-cloud-native-ecosystem) goals. + +It will show evidence to the community that the project supports multiple [platform interface specification](https://github.com/buildpacks/spec/blob/main/platform.md) implementers increasing community's confidence on the flexibility of specification maintained by the [CNB](https://buildpacks.io/) project. + +It will help the [CNB](https://buildpacks.io/) community (+550 members on slack channel) to grow by adding all the [kpack](https://github.com/pivotal/kpack/) community into [CNB](https://buildpacks.io/) space. + +[CNB](https://buildpacks.io/) is part of the [Cloud Native Computing Foundation](https://www.cncf.io), an open source, vendor neutral hub of cloud native computing projects, the inclusion of [kpack](https://github.com/pivotal/kpack/) under this umbrella will provide more opportunity to the community: + +- Increase in adopters, users looking to use buildpacks in [Kubernetes](https://kubernetes.io/) will find a tool supported and maintained by the [CNB team](https://github.com/buildpacks/community/blob/main/TEAMS.md). +- Improve efficiency, ensuring that the roadmaps of the two projects are closer aligned will make it easier to coordinate efforts between both communities. + +### What use cases does it support? + +[kpack](https://github.com/pivotal/kpack/) will add support to operators by providing declarative [Kubernetes](https://kubernetes.io/) resources (images, builders, or stacks for example) to monitor for security patches on the underlying builder's buildpacks or stacks and rebuild the OCI image when changes are detected, allowing platforms to roll out new versions of the applications when vulnerabilities are fixed. + +### How does kpack support the goals and use cases of the project? + +The [CNB](https://buildpacks.io/) project turns application source code into OCI-compliant container images; in order to do that, it defines a platform-to-buildpack contract that guarantees interoperability between different implementers. + +The [CNB](https://buildpacks.io/) project embraces modern container standards, and [Kubernetes](https://kubernetes.io/) has become the industry standard for automating deployment, scaling, and management of containerized applications. + +[kpack](https://github.com/pivotal/kpack/) fits perfectly in that direction because it implements the [platform interface specification](https://github.com/buildpacks/spec/blob/main/platform.md) and because is a [Kubernetes](https://kubernetes.io/) native application its community possesses a vast knowledge that can provide valuable feedback to the CNB project. + +### Is there functionality in kpack that is already provided by the project? + +[pack](https://github.com/buildpacks/pack) and [kpack](https://github.com/pivotal/kpack/) offer similar functionality (both tools implement the [platform interface](https://github.com/buildpacks/spec/blob/main/platform.md)[specification](https://github.com/buildpacks/spec/blob/main/platform.md)) but they do it for two non-overlapping contexts: while the first one targets developers and local builds, [kpack](https://github.com/pivotal/kpack/) manages containerization on day-2 and at scale and is a [Kubernetes](https://kubernetes.io/) native implementation. + +### Is kpack integrated with another service or technology that is widely used? + +As mentioned earlier, [kpack](https://github.com/pivotal/kpack/) implements the [platform interface specification](https://github.com/buildpacks/spec/blob/main/platform.md) on [Kubernetes](https://kubernetes.io/), a standard nowadays for automating deployment, scaling, and management of containerized applications. + +# What it is + +[Kubernetes](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) is a portable, extensible, open-source platform for managing containerized workloads and services. The [Kubernetes](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) API can be extended in different ways; one of them is using [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/), a custom resource represents a customization of a particular [Kubernetes](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) installation. + +[kpack](https://github.com/pivotal/kpack/) extends [Kubernetes](https://kubernetes.io/) using [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) and utilizes unprivileged [Kubernetes](https://kubernetes.io/) primitives to provide builds of OCI images as a platform implementation of [Cloud Native Buildpacks](https://buildpacks.io/). This means that [kpack](https://github.com/pivotal/kpack/) takes the CNB-defined concepts (image, builder, stacks, etc) and bakes them into the Kubernetes extension model using custom resources and exposing a declarative API for interacting with it. + +The declarative API enforces a separation of responsibilities. Operators declare the configuration for a CNB image or define which buildpacks or stacks must be used, and [kpack](https://github.com/pivotal/kpack/) - using its custom controller - will take care of the heavy lifting, keeping the state of the custom objects in sync with the declared desired state. + +# How it Works + +As mentioned before, [kpack](https://github.com/pivotal/kpack/) uses the [custom resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) extension point to provide the capabilities of building OCI images as a platform implementation of [Cloud Native Buildpacks](https://buildpacks.io/). + +These custom resources have a common definition similar to this: + +```yaml +apiVersion: kpack.io/v1alpha2 +kind: [ClusterStack|ClusterStore|Image|Builder|Build] +metadata: + name: [unique name] +``` + +The _apiVersion_ key specifies which version of the Kubernetes API is used to create the object, in this case **kpack.io/v1alpha2** + +The _kind_ key specifies what kind of objects we want to create for example: **ClusterStack, ClusterStore, Image, Builder or Build** + +The _metadata_ key is used to define the data that can uniquely identify the object. One common key used around all the custom resources is to provide a _name_ to identify the object. + +Some of the [custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) implemented by [kpack](https://github.com/pivotal/kpack/) are describe in the next section, if you want to check the complete reference go to [kpack](https://github.com/pivotal/kpack/) documentation [site](https://github.com/pivotal/kpack/tree/main/docs) + +## ClusterStack + +This resource is an abstraction to group a `build image` and a `run image` required to build the application source code. + +Let's see an example of the [ClusterStack](https://github.com/pivotal/kpack/blob/main/docs/stack.md) definition + +```yaml +apiVersion: kpack.io/v1alpha2 +kind: ClusterStack +metadata: + name: base +spec: + id: "io.buildpacks.stacks.bionic" + buildImage: + image: "my-buildpack-repo/build:cnb" + runImage: + image: "my-buildpack-repo/run:cnb" +``` + +The _spec_ key is used to define the desired state of the ClusterStack and the keys availables under _spec_ match the values expected in a CNB [stack](https://buildpacks.io/docs/concepts/components/stack/) definition: + +- _id_: The 'id' of the stack +- _buildImage.image_: The `build-image` of the [stack](https://buildpacks.io/docs/concepts/components/stack/). +- _runImage.image_: The `run-image` of the [stack](https://buildpacks.io/docs/concepts/components/stack/). + +## Cluster Store + +Creates a repository of buildpacks packaged as OCI artifacts to be used during a build. + +Let's see an example of the [ClusterStore](https://github.com/pivotal/kpack/blob/main/docs/store.md) definition + +``` yaml +apiVersion: kpack.io/v1alpha2 +kind: ClusterStore +metadata: + name: my-cluster-store +spec: + sources: + - image: foo.com/my-buildpack-repo/buildpack-1@sha256:sha123 + - image: foo.com/my-buildpack-repo/buildpack-2@sha256:sha345 + - image: foo.com/my-buildpack-repo/builder:base + ``` + +The _spec_ key is used to define the desired state of the [ClusterStore](https://github.com/pivotal/kpack/blob/main/docs/store.md) + +- _sources_: List of buildpackage images to make available in the ClusterStore. Each image is an object with the key _image_. + +As a side note the [ClusterStore](https://github.com/pivotal/kpack/blob/main/docs/store.md) resource will be deprecated in favor of a new Buildpack resource in the near future according to the following [RFC](https://www.google.com/url?q=https://github.com/pivotal/kpack/pull/931&sa=D&source=docs&ust=1665521917723122&usg=AOvVaw1eNN-XzLf5xiX1nvrHKMRE) + +## Builder or ClusterBuilder + +Creates a [CNB builder](https://buildpacks.io/docs/concepts/components/builder/) image that contains all the components necessary to execute a build. + +An example of the [Builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md) definition is as follows: + +```yaml +apiVersion: kpack.io/v1alpha2 +kind: Builder +metadata: + name: my-builder +spec: + tag: foo.com/sample/builder + stack: + name: base + kind: ClusterStack + store: + name: my-cluster-store + kind: ClusterStore + order: + - group: + - id: my-buildpack-repo/buildpack-1 + - group: + - id: my-buildpack-repo/buildpack-2 + ``` + +It's important to notice that a [ClusterStack](https://github.com/pivotal/kpack/blob/main/docs/stack.md) and [ClusterStore](https://github.com/pivotal/kpack/blob/main/docs/store.md) is required for creating a [Builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md). + +The _spec_ key is used to define the desired state of the [Builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md) + +- _tag_: The tag to save the builder image. +- _stack.name_: The name of the stack resource to use as the builder stack. All buildpacks in the order must be compatible with the [ClusterStack](https://github.com/pivotal/kpack/blob/main/docs/stack.md). +- _stack.kind_: The type as defined in [Kubernetes](https://kubernetes.io/). This will always be [ClusterStack](https://github.com/pivotal/kpack/blob/main/docs/stack.md). +- _store.name_: The name of the [ClusterStore](https://github.com/pivotal/kpack/blob/main/docs/store.md) resource in [Kubernetes](https://kubernetes.io/). +- _store.kind_: The type as defined in [Kubernetes](https://kubernetes.io/). This will always be [ClusterStore](https://github.com/pivotal/kpack/blob/main/docs/store.md). +- _order_: The [builder order](https://buildpacks.io/docs/reference/builder-config/). + +The [ClusterBuilder](https://github.com/pivotal/kpack/blob/main/docs/builders.md#cluster-builders) resource is almost identical to a [Builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md) but it is a cluster scoped resource that can be referenced by an [Image](https://github.com/pivotal/kpack/blob/main/docs/image.md) in any namespace. + +## Build + +Custom resource responsible for scheduling and running a single build. + +An example of a [Build](https://github.com/pivotal/kpack/blob/main/docs/build.md) definition is + +```yaml +apiVersion: kpack.io/v1alpha2 +kind: Build +metadata: + name: sample-build +spec: + tags: + -sample/image + builder: + image: foo.com/sample/builder + projectDescriptorPath: path/to/project.toml + source: + git: + url: https://github.com/my-account/sample-app.git + revision: main +``` + +The _spec_ key is used to define the desired state of the [Build](https://github.com/pivotal/kpack/blob/main/docs/build.md) + +- _tags_: A list of tags to build. At least one tag is required. +- _builder.image_: This is the tag to the [Cloud Native Buildpacks builder image](https://buildpacks.io/docs/concepts/components/builder/) to use in the build. +- _source_: The source location that will be the input to the build. +- _projectDescriptorPath_: Path to the [project descriptor file](https://buildpacks.io/docs/reference/config/project-descriptor/) relative to source root dir or subPath if set. + +## Image + +Provides a configuration to build and maintain an OCI image utilizing [CNB](https://buildpacks.io/). + +An example of an [Image](https://github.com/pivotal/kpack/blob/main/docs/image.md) definition is as follows + +```yaml +apiVersion: kpack.io/v1alpha2 +kind: Image +metadata: + name: my-app-image + namespace: default +spec: + tag: foo.com/my-app-repo/my-app-image + builder: + name: my-builder + kind: Builder + source: + git: + url: https://github.com/my-account/sample-app.git + revision: 82cb521d636b282340378d80a6307a08e3d4a4c4 +``` + +The _spec_ key is used to define the desired state of the [Image](https://github.com/pivotal/kpack/blob/main/docs/image.md) + +- _tag_: The image tag. +- _builder_: Configuration of the [builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md) resource the image builds will use. +- source: The source code that will be monitored/built into images. + +# Migration + +### Repositories + +The suggested strategy for migrating [kpack's](https://github.com/pivotal/kpack/) git repositories to the [CNB](https://buildpacks.io/) is to use the [transfer repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository#transferring-a-repository-owned-by-your-organization) git feature. + +The following table shows the candidates repositories to be transferred + +| Origin Repo | Description | Owner | Destination Repo | Owner | +| --- | --- | --- | --- | --- | +| [https://github.com/pivotal/kpack](https://github.com/pivotal/kpack) | kpack source code | Pivotal | [https://github.com/buildpacks/kpack](https://github.com/buildpacks/kpack) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | +| [https://github.com/vmware-tanzu/kpack-cli](https://github.com/vmware-tanzu/kpack-cli) | kpack CLI | VMware | [https://github.com/buildpacks/kpack-cli](https://github.com/buildpacks/kpack-cli) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | +| [https://github.com/vmware-tanzu/homebrew-kpack-cli](https://github.com/vmware-tanzu/homebrew-kpack-cli) | Homebrew tap for the kpack CLI | VMware | [https://github.com/buildpacks/homebrew-kpack-cli](https://github.com/buildpacks/homebrew-kpack-cli) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | + +For each repository + +- The owner or admin user must follow the steps describe in github [documentation](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository#transferring-a-repository-owned-by-your-organization) and transfer the repository to the organization [Cloud Native Buildpacks](https://github.com/buildpacks) +- A member of the [TOC team](https://github.com/orgs/buildpacks/teams/toc/members) in [CNB](https://buildpacks.io/) must accept the donation of the repository. The name of the destination repository will be the one described in the table above. + +### CI / CD Pipelines + +[kpack's](https://github.com/pivotal/kpack/) CI/CD infrastructure currently runs on internal VMware infrastructure. [kpack's](https://github.com/pivotal/kpack/) CI/CD pipelines will need to be rebuilt on [CNB](https://buildpacks.io/) infrastructure. + +### Documentation + +[Kpack](https://github.com/pivotal/kpack/) documentation is currently hosted in the base code [repository](https://github.com/pivotal/kpack/tree/main/docs), after migrating to [CNB](https://buildpacks.io/) the documentation will be published into the Cloud Native Buildpack [site](https://buildpacks.io/). + +[CNB](https://buildpacks.io/) already mentioned [kpack](https://github.com/pivotal/kpack/) in their documentation, specifically, in the tools section. The proposal is: + +- Create a new folder name **kpack** inside the [tool](https://github.com/buildpacks/docs/tree/main/content/docs/tools) section in the docs repository +- Copy kpack's [documentation](https://github.com/pivotal/kpack/tree/main/docs) into this new created folder +- Update the references and all the required elements to format the documentation according to [CNB](https://buildpacks.io/) site + +### Governance + +Based on the [CNB governance policy](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md) and the fact that [kpack](https://github.com/pivotal/kpack/) is a [platform](https://buildpacks.io/docs/concepts/components/platform/) implementation of [Cloud Native Buildpacks](https://buildpacks.io/), it will be added under the responsibility of the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team). + +How do migrate roles and responsibilities into the CNB governance process? + +Currently the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team) already has a **team lead** assigned and, by definition, each team can have only one **team lead**. In order to provide the current [kpack](https://github.com/pivotal/kpack/) team with the same accountability for the migrated repositories the proposal is to follow the guidelines describe on the [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) + +# Risks + +- It's not clear how to handle the budget required to finance the infrastructure to rebuild the CI/CD pipelines on CNCF CNB infrastructure. +- Evaluate any legal requirement from [CNCF](https://www.cncf.io) that must be fulfilled before accepting the project into the [CNB](https://buildpacks.io/) ecosystem. +- Relying on the approval of [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) to guarantee current [kpack](https://github.com/pivotal/kpack/) maintainers could keep supporting the project after the donation. + +# Drawbacks + +Why should we _not_ do this? + +- If the [CNB](https://buildpacks.io/) team expects to implement a different kind of integration with [Kubernetes](https://kubernetes.io/), then accepting the donation of [kpack](https://github.com/pivotal/kpack/) could conflict with that strategy. +- Another component to maintain which requires additional context and expertise in [Kubernetes](https://kubernetes.io/). + +# Alternatives + +- What other designs have been considered? + - [VMware](https://www.vmware.com/) could continue to control the project, but it doesn't help on increase the adoption because it remains as a single-vendor driven project + - [VMware](https://www.vmware.com/) could donate [kpack](https://github.com/pivotal/kpack/) to the [Continuous Delivery Foundation](https://cd.foundation/), but [CNB](https://buildpacks.io/) presents a natural home for [kpack](https://github.com/pivotal/kpack/) (it is an implementation of the platform specification) + - [VMware](https://www.vmware.com/) could create a new [CNCF](https://www.cncf.io/) project and move all [kpack](https://github.com/pivotal/kpack/) resources to it, but in this case it would need to undergo as a sandbox project for example. + +- Why is this proposal the best? + +[kpack](https://github.com/pivotal/kpack/) is a mature Kubernetes-native tool that leverages buildpacks and is used in production environments. The project's maintainers and contributors possess valuable technical and user context, derived from developing [kpack](https://github.com/pivotal/kpack/) and integrating feedback from users utilizing [CNB](https://buildpacks.io/) concepts when presented as part of Kubernetes resources. + +- What is the impact of not doing this? + +The [CNB](https://buildpacks.io/) community would have to develop from scratch any kind of integration with the Cloud Native Ecosystem to satisfy the project goals. + +**Prior Art** + +- Guidelines for accepting component-level contributions [RFC #143](https://github.com/buildpacks/rfcs/pull/143) +- Component Maintainer Role [RFC #234](https://github.com/buildpacks/rfcs/pull/234) + +# Unresolved Questions + +See the risks section + +# Spec. Changes (OPTIONAL) + +None From 5ebc55eb9de8611300805681852271105325a046 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 14 Oct 2022 00:03:13 +0000 Subject: [PATCH 172/372] Bump kentaro-m/auto-assign-action from 1.2.1 to 1.2.4 Bumps [kentaro-m/auto-assign-action](https://github.com/kentaro-m/auto-assign-action) from 1.2.1 to 1.2.4. - [Release notes](https://github.com/kentaro-m/auto-assign-action/releases) - [Commits](https://github.com/kentaro-m/auto-assign-action/compare/v1.2.1...v1.2.4) --- updated-dependencies: - dependency-name: kentaro-m/auto-assign-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- .github/workflows/auto-assign-maintainer.yml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/auto-assign-maintainer.yml b/.github/workflows/auto-assign-maintainer.yml index a247c7174..80b2ec846 100644 --- a/.github/workflows/auto-assign-maintainer.yml +++ b/.github/workflows/auto-assign-maintainer.yml @@ -14,7 +14,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.1 + - uses: kentaro-m/auto-assign-action@v1.2.4 with: configuration-path: .github/auto-assign-core.yml distribution: @@ -23,7 +23,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.1 + - uses: kentaro-m/auto-assign-action@v1.2.4 with: configuration-path: .github/auto-assign-distribution.yml implementation: @@ -32,7 +32,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.1 + - uses: kentaro-m/auto-assign-action@v1.2.4 with: configuration-path: .github/auto-assign-implementation.yml learning: @@ -41,7 +41,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.1 + - uses: kentaro-m/auto-assign-action@v1.2.4 with: configuration-path: .github/auto-assign-learning.yml platform: @@ -50,7 +50,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.1 + - uses: kentaro-m/auto-assign-action@v1.2.4 with: configuration-path: .github/auto-assign-platform.yml bat: @@ -59,6 +59,6 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.1 + - uses: kentaro-m/auto-assign-action@v1.2.4 with: configuration-path: .github/auto-assign-bat.yml From a8aed9eec457ecde45ee73e254ca660cad831bf0 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 18 Oct 2022 18:15:10 -0500 Subject: [PATCH 173/372] WIP - adding slack channel and RFC sections Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 48 ++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 2fd41ec7f..f62cabc16 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -265,12 +265,59 @@ For each repository ### Governance +#### Team roles + Based on the [CNB governance policy](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md) and the fact that [kpack](https://github.com/pivotal/kpack/) is a [platform](https://buildpacks.io/docs/concepts/components/platform/) implementation of [Cloud Native Buildpacks](https://buildpacks.io/), it will be added under the responsibility of the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team). How do migrate roles and responsibilities into the CNB governance process? Currently the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team) already has a **team lead** assigned and, by definition, each team can have only one **team lead**. In order to provide the current [kpack](https://github.com/pivotal/kpack/) team with the same accountability for the migrated repositories the proposal is to follow the guidelines describe on the [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) +#### RFC process + +Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will follow the [RFC process](https://github.com/buildpacks/rfcs) stablished in CNB project for any new RFC required by the project. + +##### Template + +[kpack](https://github.com/pivotal/kpack/) have their own RFC process, let's compare both templates + +| [kpack template](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) | [CNB template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) mapping | +| ----------------------- | ------------------------------ | +| Problem | Motivation | +| Outcome | Motivation | +| Actions to take | What it is / How it works | +| Complexity | - | +| - | Migration | +| Prior Art | Prior Art | +| Alternatives | Alternatives | +| Risks | Drawbacks | +| - | Unresolved Questions | +| - | Spec. Changes | +| - | History| + +As we can see the [CNB RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) consider most of the sections required by the [kpack RFC](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) process, the only one missing is: + - **complexity**: What thoughts do you have on the complexity of this? + +The proposal is to incorporate this section into the [CNB template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md). + +##### Existing RFC + +- Open: +- Closed: + +#### Slack channel + +The proposal is to move the [kpack](https://github.com/pivotal/kpack/) slack instance from the [Kubernetes slack instance](https://kubernetes.slack.com/channels/kpack) to the [CNCF slack instance](https://slack.cncf.io/). + +[kpack](https://github.com/pivotal/kpack/) maintainers will have to request the migration of the [kpack](https://github.com/pivotal/kpack/) slack over to [CNCF slack](https://slack.cncf.io/) and co-ordinate the announcements/user migration. + +The following are the slack channels that will be migrated over and their proposed names after migrations + +| Current | Proposed | +| ----------------------- | ------------------------------ | +| kpack | buildpacks-kpack | + + # Risks - It's not clear how to handle the budget required to finance the infrastructure to rebuild the CI/CD pipelines on CNCF CNB infrastructure. @@ -303,6 +350,7 @@ The [CNB](https://buildpacks.io/) community would have to develop from scratch a - Guidelines for accepting component-level contributions [RFC #143](https://github.com/buildpacks/rfcs/pull/143) - Component Maintainer Role [RFC #234](https://github.com/buildpacks/rfcs/pull/234) +- Proposal to move CNCF slack [RFC #198](https://github.com/buildpacks/rfcs/pull/198) # Unresolved Questions From 71efdee835f654a964f4c1f7b7b836078c6357b5 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 19 Oct 2022 09:51:06 -0500 Subject: [PATCH 174/372] adding RFC process proposal Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 31 +++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index f62cabc16..f9581e38b 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -253,6 +253,13 @@ For each repository [kpack's](https://github.com/pivotal/kpack/) CI/CD infrastructure currently runs on internal VMware infrastructure. [kpack's](https://github.com/pivotal/kpack/) CI/CD pipelines will need to be rebuilt on [CNB](https://buildpacks.io/) infrastructure. +##### Hardware requirements + +The minimal hardware requirements to request to CNCF to recreate the CI/CD pipelines are: + +- TBD + + ### Documentation [Kpack](https://github.com/pivotal/kpack/) documentation is currently hosted in the base code [repository](https://github.com/pivotal/kpack/tree/main/docs), after migrating to [CNB](https://buildpacks.io/) the documentation will be published into the Cloud Native Buildpack [site](https://buildpacks.io/). @@ -275,13 +282,24 @@ Currently the [CNB Platform Team](https://github.com/buildpacks/community/blob/m #### RFC process -Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will follow the [RFC process](https://github.com/buildpacks/rfcs) stablished in CNB project for any new RFC required by the project. +Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will follow the [RFC process](https://github.com/buildpacks/rfcs) stablished in CNB project for any new RFC created in the project. + +##### Existing RFC + +- Open: Currently there are < 10 open RFCs (some of them opened 2 years ago). + - The proposal is to suggest the [kpack](https://github.com/pivotal/kpack/) maintainers to: + - Triage those RFCs an update their status before the donation. + - Co-ordinate the announcement of the donation to the RFCs authors and explain them the strategy after the migration (next section) + - After the donation, any open RFCs in [kpack](https://github.com/pivotal/kpack/) repository should be closed + - The RFC author should create a new RFC in the CNB RFC [repository](https://github.com/buildpacks/rfcs) and follow the CNB [RFC process](https://github.com/buildpacks/rfcs) + +- Closed: For historical purpose, we will keep those RFC in the repo ##### Template -[kpack](https://github.com/pivotal/kpack/) have their own RFC process, let's compare both templates +The following table shows the mapping between the sections of the current kpack [RFC template](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) vs the CNB [RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) -| [kpack template](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) | [CNB template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) mapping | +| kpack template | CNB template | | ----------------------- | ------------------------------ | | Problem | Motivation | | Outcome | Motivation | @@ -295,16 +313,11 @@ Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will | - | Spec. Changes | | - | History| -As we can see the [CNB RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) consider most of the sections required by the [kpack RFC](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) process, the only one missing is: +As we can see the CNB [RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) consider most of the sections required by the kpack [RFC](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) process, the only missing one is: - **complexity**: What thoughts do you have on the complexity of this? The proposal is to incorporate this section into the [CNB template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md). -##### Existing RFC - -- Open: -- Closed: - #### Slack channel The proposal is to move the [kpack](https://github.com/pivotal/kpack/) slack instance from the [Kubernetes slack instance](https://kubernetes.slack.com/channels/kpack) to the [CNCF slack instance](https://slack.cncf.io/). From 99041240b0ef9bac2c73b809958c8d6da5fc1ffd Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 19 Oct 2022 09:54:34 -0500 Subject: [PATCH 175/372] adding RFC process proposal Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index f9581e38b..fb63c677f 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -286,14 +286,14 @@ Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will ##### Existing RFC -- Open: Currently there are < 10 open RFCs (some of them opened 2 years ago). +- **Open**: Currently there are less that 10 [open RFCs](https://github.com/pivotal/kpack/pulls?q=is%3Apr+label%3ARFC+is%3Aopen) (some of them opened 2 years ago) in [kpack](https://github.com/pivotal/kpack/) repository. - The proposal is to suggest the [kpack](https://github.com/pivotal/kpack/) maintainers to: - Triage those RFCs an update their status before the donation. - Co-ordinate the announcement of the donation to the RFCs authors and explain them the strategy after the migration (next section) - After the donation, any open RFCs in [kpack](https://github.com/pivotal/kpack/) repository should be closed - The RFC author should create a new RFC in the CNB RFC [repository](https://github.com/buildpacks/rfcs) and follow the CNB [RFC process](https://github.com/buildpacks/rfcs) -- Closed: For historical purpose, we will keep those RFC in the repo +- **Closed**: For historical purpose, we will keep those RFC in the repository. ##### Template From 3cee2927a11f57a40f5810006f62ea8df26a679a Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 19 Oct 2022 11:12:47 -0500 Subject: [PATCH 176/372] adding cosing example Signed-off-by: Juan Bustamante --- ...00-governance-component-maintainer-role.md | 38 +++++++++++-------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index 563360072..720b0da2c 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -59,7 +59,9 @@ As we can see in diagram above, a CNB team takes care of `N` number of [software Depending on the team, these components can increase in size or complexity, or there could be someone from the community that wants to specialize their contributions on certain components without taking the responsibility of become a **team maintainer**. -### Example +### Examples + +#### Platform Team Let's take for example the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team), which right now have 2 maintainers, and let's use the LOC (lines of code) metrics, for each of the components maintains by this team, to dimension the size of it. @@ -71,20 +73,12 @@ Let's take for example the [Platform Team](https://github.com/buildpacks/communi **Note**: [Tokei](https://github.com/XAMPPRocky/tokei) tool was used to calculate the LOC of the repositories. -#### Integration with the Cloud Native Ecosystem +##### Integration with the Cloud Native Ecosystem As part of the [CNB roadmap](https://github.com/buildpacks/community/blob/main/ROADMAP.md#integration-with-the-cloud-native-ecosystem) a `better out-of-the box Kubernetes and Docker integration` is a goal of the project and in order to do that, the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team) will have to include another software component that accomplish this goal. -In case of this scenario, then an hypothetical update of the component table will be: - -| Component | LOC | -|--------------------------|--------| -| [Pack]() | +58000 | -| [Tekton Tasks + Pipelines]() | +2150 | -| [CircleCI Pack Orb]() | +400 | -| Kubernetes Native Platform implementation | ? | +In case of this scenario, then an hypothetical update of the component table if kpack project is donated into CNB (see [RFC](https://github.com/buildpacks/rfcs/pull/235) for more details) will be: -Because, this is hypothetical scenario, we actually don't have the size but, we can use [kpack]() as reference, because it is a Kubernetes Platform implementation of the CNB specification. Let's update the table again. | Component | LOC | |--------------------------|--------| @@ -96,6 +90,10 @@ Because, this is hypothetical scenario, we actually don't have the size but, we As we can see, a new implementation of the [Platform Interface Specification](https://github.com/buildpacks/spec/blob/main/platform.md) could be as big as [pack]() but most important: - It requires a specific knowledge in [Kubernetes](https://kubernetes.io/) +##### Adding Consign integration + +Another example of problem presented in this RFC is: [adding support to consign RFC](https://github.com/buildpacks/rfcs/pull/195). In this RFC a new phase executable must be developed and maintain by the Platform team, but this implementation requires knowledge and expertise on technologies like [sigstore](https://www.sigstore.dev/) + # How it Works [how-it-works]: #how-it-works @@ -111,13 +109,23 @@ An updated version of the previous diagram shows graphically this new roles ![](https://i.imgur.com/rWElkCw.png) -### Example +### Examples + +Let's come back to our previous examples. + +#### Platform Team + +In this cases, the existence of the **component maintainer** role will provide the rules to the platform team lead or platform maintainers to nominate (following the guidelines describe in the next section) a **component maintainer**. + +##### Integration with the Cloud Native Ecosystem + +In case [kpack](https://github.com/pivotal/kpack) is donated to CNB, [kpack](https://github.com/pivotal/kpack) maintainers could become **component maintainers** of this component and keep doing all the activities required for maintaining the lights on in the project without having to assume the whole set of responsibilities of a team maintainer. Also, platform maintainers will not be overwhelm being the sole reviewers/approvers for PRs for [kpack](https://github.com/pivotal/kpack) if they are not familiar with this project. -Let's come back to our previous [kpack](https://github.com/pivotal/kpack) example. +##### Adding Consign integration -#### Integration with the Cloud Native Ecosystem +The existence of the **component maintainer** will open the door to the community, in particular, those volunteers with experience on [sigstore](https://www.sigstore.dev/) to help on with the contributions and maintenance of the new `signer` binary proposed. -In this case, the existence of the **component maintainer** role will provide the rules to the **platform team lead** or **platform maintainers** to nominate (following the guidelines describe in the next section) a **component maintainer** for [kpack](https://github.com/pivotal/kpack) when they consider is necessary. +This case, is an example of two different areas of interest or knowledge where having the separation of responsibilities is relevant. ## Guidelines nominate component maintainer From 96202b1a84c22f56e64e08ba8c1319e09d368d40 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 21 Oct 2022 11:22:20 -0500 Subject: [PATCH 177/372] Removing the template RFC section after Matthew comments Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index fb63c677f..2fc098305 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -282,7 +282,7 @@ Currently the [CNB Platform Team](https://github.com/buildpacks/community/blob/m #### RFC process -Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will follow the [RFC process](https://github.com/buildpacks/rfcs) stablished in CNB project for any new RFC created in the project. +Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will follow the [RFC process](https://github.com/buildpacks/rfcs) and [RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) stablished in CNB project for any new RFC created in the project. ##### Existing RFC @@ -295,29 +295,6 @@ Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will - **Closed**: For historical purpose, we will keep those RFC in the repository. -##### Template - -The following table shows the mapping between the sections of the current kpack [RFC template](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) vs the CNB [RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) - -| kpack template | CNB template | -| ----------------------- | ------------------------------ | -| Problem | Motivation | -| Outcome | Motivation | -| Actions to take | What it is / How it works | -| Complexity | - | -| - | Migration | -| Prior Art | Prior Art | -| Alternatives | Alternatives | -| Risks | Drawbacks | -| - | Unresolved Questions | -| - | Spec. Changes | -| - | History| - -As we can see the CNB [RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) consider most of the sections required by the kpack [RFC](https://github.com/pivotal/kpack/blob/main/rfcs/0000-template.md) process, the only missing one is: - - **complexity**: What thoughts do you have on the complexity of this? - -The proposal is to incorporate this section into the [CNB template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md). - #### Slack channel The proposal is to move the [kpack](https://github.com/pivotal/kpack/) slack instance from the [Kubernetes slack instance](https://kubernetes.slack.com/channels/kpack) to the [CNCF slack instance](https://slack.cncf.io/). From 2e33f4f5f39863019ae93a3d4e22eeb728a3180a Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 21 Oct 2022 12:31:54 -0500 Subject: [PATCH 178/372] adding feedback into the slack channel section Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 2fc098305..e513ed39d 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -297,16 +297,13 @@ Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will #### Slack channel -The proposal is to move the [kpack](https://github.com/pivotal/kpack/) slack instance from the [Kubernetes slack instance](https://kubernetes.slack.com/channels/kpack) to the [CNCF slack instance](https://slack.cncf.io/). +The proposals are: + - `keep` the [kpack](https://github.com/pivotal/kpack/) slack instance from the [Kubernetes slack instance](https://kubernetes.slack.com/channels/kpack), as [kpack](https://github.com/pivotal/kpack/) is a Kubernetes native application most of their users already use [Kubernetes slack instance](https://kubernetes.slack.com/channels/kpack) for communication. + - `create` a new channel in the [CNCF slack instance](https://slack.cncf.io/), this will bring the two communities (kpack and CNB) together -[kpack](https://github.com/pivotal/kpack/) maintainers will have to request the migration of the [kpack](https://github.com/pivotal/kpack/) slack over to [CNCF slack](https://slack.cncf.io/) and co-ordinate the announcements/user migration. - -The following are the slack channels that will be migrated over and their proposed names after migrations - -| Current | Proposed | -| ----------------------- | ------------------------------ | -| kpack | buildpacks-kpack | +[kpack](https://github.com/pivotal/kpack/) maintainers should include the notification of the new channel in the announcement of the donation. +[Platform maintainers](https://github.com/buildpacks/community/blob/main/TEAMS.md#maintainers-1) will have to request or create the new slack channel with the following name: **buildpacks-kpack** (which will be defined as the preferred channel to be used). # Risks From 74ad6dd4ad7a2d6351a6b4b2f886878f32b6e88a Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 24 Oct 2022 16:19:51 -0500 Subject: [PATCH 179/372] adding more detail to how it works section Signed-off-by: Juan Bustamante --- ...00-governance-component-maintainer-role.md | 25 ++++++++++++++++--- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index 720b0da2c..003671845 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -23,6 +23,7 @@ This RFC proposes the creation of a new role **component maintainer** in the pro - **team lead**: is a maintainer who has special responsibilities for representing team concerns at the project level - **component maintainer**: the proposed role in this RFC. Individual that maintain specific code repositories of a software-component inside a CNB team. - **software component**: is a software unit with a well-defined interface and explicitly specified dependencies. +- **github team**: groups of organization members that reflects a company or group's structure with cascading access permissions. # Motivation [motivation]: #motivation @@ -90,14 +91,14 @@ In case of this scenario, then an hypothetical update of the component table if As we can see, a new implementation of the [Platform Interface Specification](https://github.com/buildpacks/spec/blob/main/platform.md) could be as big as [pack]() but most important: - It requires a specific knowledge in [Kubernetes](https://kubernetes.io/) -##### Adding Consign integration +##### Adding Cosign integration Another example of problem presented in this RFC is: [adding support to consign RFC](https://github.com/buildpacks/rfcs/pull/195). In this RFC a new phase executable must be developed and maintain by the Platform team, but this implementation requires knowledge and expertise on technologies like [sigstore](https://www.sigstore.dev/) # How it Works [how-it-works]: #how-it-works -The proposal is to incorporate a **component maintainer** role. +The proposal is to incorporate a **component maintainer** role in our governance process. The **component maintainer** will take under his/her responsibility a well defined software component in a CNB team and for each repository will be allow to: @@ -105,10 +106,26 @@ The **component maintainer** will take under his/her responsibility a well defin - planning release milestones, and releasing components versions - edit, label, assign issues -An updated version of the previous diagram shows graphically this new roles +An updated version of the previous diagram shows graphically this new role. ![](https://i.imgur.com/rWElkCw.png) +As we can see in the previous diagram, a new orange box was drawn representing a component or group of software components the **component maintainer** is taking responsibilities for. When a team lead or maintainer consider to use the role in their team, they must follow these steps: + +- In case it doesn't exist, a new **github team** must be created. A recommended name for this new team must follows the format `[cnb-team]-[component]-maintainers`, where: + - **cnb-team**: is the CNB team responsible for this software components, for example: Platform Team. + - **component**: is the software component name. for example: kpack. + + Some examples are: `platform-kpack-maintainers` or `platform-cosign-maintainers` + +- For each repository related to the component or group of components: + - In case it doesn't exist, a new **CODEOWNERS** file must be added. + - Add the team `[cnb-team]-[component]-maintainers` created into the **CODEOWNERS** file. + - Members of the team should have maintainers permissions. + - The branch protection should require **CODEOWNERS** to approve or merge a pull request. + +- Add the new **component maintainer** into the `[cnb-team]-[component]-maintainers` **github team**. + ### Examples Let's come back to our previous examples. @@ -121,7 +138,7 @@ In this cases, the existence of the **component maintainer** role will provide t In case [kpack](https://github.com/pivotal/kpack) is donated to CNB, [kpack](https://github.com/pivotal/kpack) maintainers could become **component maintainers** of this component and keep doing all the activities required for maintaining the lights on in the project without having to assume the whole set of responsibilities of a team maintainer. Also, platform maintainers will not be overwhelm being the sole reviewers/approvers for PRs for [kpack](https://github.com/pivotal/kpack) if they are not familiar with this project. -##### Adding Consign integration +##### Adding Cosign integration The existence of the **component maintainer** will open the door to the community, in particular, those volunteers with experience on [sigstore](https://www.sigstore.dev/) to help on with the contributions and maintenance of the new `signer` binary proposed. From 5882ea5b27143099b33f8590c20dc7d5efbe5d37 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 25 Oct 2022 09:02:50 -0500 Subject: [PATCH 180/372] fixing some issues with the grammar Signed-off-by: Juan Bustamante --- ...00-governance-component-maintainer-role.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0000-governance-component-maintainer-role.md index 003671845..f0a1ca388 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0000-governance-component-maintainer-role.md @@ -56,7 +56,7 @@ CNB maintainer contributor responsibility can be describe as: `is in charge of t ![](https://i.imgur.com/yXsLK6N.png) -As we can see in diagram above, a CNB team takes care of `N` number of [software components](#definitions) and the project contributors from the community make them contributions to any of those components. +As we can see in diagram above, a CNB team takes care of `N` number of [software components](#definitions) and the project contributors from the community make their contributions to any of those components. Depending on the team, these components can increase in size or complexity, or there could be someone from the community that wants to specialize their contributions on certain components without taking the responsibility of become a **team maintainer**. @@ -64,7 +64,7 @@ Depending on the team, these components can increase in size or complexity, or t #### Platform Team -Let's take for example the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team), which right now have 2 maintainers, and let's use the LOC (lines of code) metrics, for each of the components maintains by this team, to dimension the size of it. +Let's take for example the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team), which right now have 2 maintainers, and let's use the LOC (lines of code) metric, for each of the components maintains by this team, to dimension the size of them. | Component | LOC | |--------------------------|--------| @@ -93,7 +93,7 @@ As we can see, a new implementation of the [Platform Interface Specification](ht ##### Adding Cosign integration -Another example of problem presented in this RFC is: [adding support to consign RFC](https://github.com/buildpacks/rfcs/pull/195). In this RFC a new phase executable must be developed and maintain by the Platform team, but this implementation requires knowledge and expertise on technologies like [sigstore](https://www.sigstore.dev/) +Another example of the problem presented in this RFC is: [adding support to cosign RFC](https://github.com/buildpacks/rfcs/pull/195). In this RFC a new phase executable must be developed and maintain by the [Platform team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team), but this implementation requires knowledge and expertise on technologies like [sigstore](https://www.sigstore.dev/). # How it Works [how-it-works]: #how-it-works @@ -110,11 +110,11 @@ An updated version of the previous diagram shows graphically this new role. ![](https://i.imgur.com/rWElkCw.png) -As we can see in the previous diagram, a new orange box was drawn representing a component or group of software components the **component maintainer** is taking responsibilities for. When a team lead or maintainer consider to use the role in their team, they must follow these steps: +As we can see a new orange box was drawn representing a component, or group of software components, the new **component maintainer** role is taking responsibilities for. When a team lead or maintainer wants to use the role in their team, they must follow these steps: - In case it doesn't exist, a new **github team** must be created. A recommended name for this new team must follows the format `[cnb-team]-[component]-maintainers`, where: - - **cnb-team**: is the CNB team responsible for this software components, for example: Platform Team. - - **component**: is the software component name. for example: kpack. + - **cnb-team**: is the CNB team responsible for the software component, for example: `Platform Team`. + - **component**: is the software component name. for example: `kpack`. Some examples are: `platform-kpack-maintainers` or `platform-cosign-maintainers` @@ -124,7 +124,7 @@ As we can see in the previous diagram, a new orange box was drawn representing a - Members of the team should have maintainers permissions. - The branch protection should require **CODEOWNERS** to approve or merge a pull request. -- Add the new **component maintainer** into the `[cnb-team]-[component]-maintainers` **github team**. +- Add the new component maintainer's github account into the `[cnb-team]-[component]-maintainers` **github team**. ### Examples @@ -132,11 +132,11 @@ Let's come back to our previous examples. #### Platform Team -In this cases, the existence of the **component maintainer** role will provide the rules to the platform team lead or platform maintainers to nominate (following the guidelines describe in the next section) a **component maintainer**. +In these cases, the existence of the **component maintainer** role will provide the rules to the platform team lead or platform maintainers to nominate (following the guidelines describe in the next section) a **component maintainer**. ##### Integration with the Cloud Native Ecosystem -In case [kpack](https://github.com/pivotal/kpack) is donated to CNB, [kpack](https://github.com/pivotal/kpack) maintainers could become **component maintainers** of this component and keep doing all the activities required for maintaining the lights on in the project without having to assume the whole set of responsibilities of a team maintainer. Also, platform maintainers will not be overwhelm being the sole reviewers/approvers for PRs for [kpack](https://github.com/pivotal/kpack) if they are not familiar with this project. +In case [kpack](https://github.com/pivotal/kpack) is donated to CNB, [kpack](https://github.com/pivotal/kpack) maintainers could become **component maintainers** of this component and keep doing all the activities required for maintaining the lights on in the project without having to assume the whole set of responsibilities of a Platform team maintainer. Also, platform maintainers will not be overwhelm being the sole reviewers/approvers for [kpack's](https://github.com/pivotal/kpack) pull requests if they are not familiar with the project. ##### Adding Cosign integration @@ -151,7 +151,7 @@ Follow this guideline to nominate a **component maintainer** for a software comp - The software component must be defined under the [GOVERNANCE](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md) team section, for example: Platform Team -> kpack - New **component maintainers** must already be contributors of the team - New **component maintainers** must be nominate by a **team lead** or a **maintainer** of the team under the following scenarios: - - A software component developed outside CNB project was [accepted](https://github.com/buildpacks/community/blob/main/contributors/guide.md#component) under their team and current team do not have the know-how or experience to handle it. + - A software component developed outside CNB project was [accepted](https://github.com/buildpacks/community/blob/main/contributors/guide.md#component) under their team and they do not have the know-how or experience to handle it. - A community **contributor** have explicitly manifest the desire to become a **component maintainer** and the **team lead** or **maintainer** consider he/she has the skills and knowledge to take the responsibility and accountability of the component. - New **component maintainers** must be elected by [super-majority](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#supermajority) of the team’s maintainers From 9f7ac8e50836bc0c59025ec17bbfe1bb47819f90 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 25 Oct 2022 10:33:33 -0500 Subject: [PATCH 181/372] adding hardware requirements Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index e513ed39d..b88fb3a12 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -257,8 +257,9 @@ For each repository The minimal hardware requirements to request to CNCF to recreate the CI/CD pipelines are: -- TBD - +###### Kubernetes cluster + - 3 nodes: 2 CPU / 8 GB RAM / 100 GB ephemeral storage per node + - At least 100 GB of storage in a public OCI registry ### Documentation From a696346207cc803f489ad4f0e7ec09cbfffaf46f Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 26 Oct 2022 15:56:14 -0500 Subject: [PATCH 182/372] updating hardware requirements for CI/CD Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index b88fb3a12..e840c81ad 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -257,9 +257,15 @@ For each repository The minimal hardware requirements to request to CNCF to recreate the CI/CD pipelines are: -###### Kubernetes cluster - - 3 nodes: 2 CPU / 8 GB RAM / 100 GB ephemeral storage per node - - At least 100 GB of storage in a public OCI registry +###### Kubernetes clusters + +**Build cluster** + +- Linux nodes + - 1 amd64 node / 2 CPU / 8GB memory / 50GB ephemeral disk storage +- Windows nodes + - 1 amd64 node / 4 CPU / 16GB memory / 100GB ephemeral disk storage +- At least 100 GB of storage in a public OCI registry ### Documentation From 067091a5d8001654b7f0895a814362f4d07ab823 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Mon, 14 Mar 2022 07:56:14 +0000 Subject: [PATCH 183/372] Propose alternative project scaffolding in pack Allow pack to promote libcnb project scaffolding and support 3rd parties to provide their own buildpack scaffolding. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 210 +++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 text/0000-pack-buildpack-new-alternatives.md diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md new file mode 100644 index 000000000..2a8be7891 --- /dev/null +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -0,0 +1,210 @@ +# Meta +[meta]: #meta +- Name: Pack Buildpack New to Support Alternatives +- Start Date: 2022-03-10 +- Author(s): aidandelaney +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal extends `pack buildpack new` with a `--libcnb` option and a `--template URL` option to allow alternatives to the shell-script template. The `--libcnb` option creates a new buildpack written in golang using `libcnb`. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. + +# Definitions +[definitions]: #definitions + +* **project scaffolding**: minimal code and file system structure used to start a project. + +# Motivation +[motivation]: #motivation + +The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal `libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. + +Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--target URL` allows other projects to re-use `pack` as their scaffolding tool. + +`pack buildpack new --libcnb` supports end-users who wish to scaffold a new `libcnb`-based buildpack. The operation of `pack buildpack new --libcnb` does not require internet access to succeed. `pack buildpack new --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. + +Extending `pack buildpack new` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. + +# What it is +[what-it-is]: #what-it-is + +Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](https://yeoman.io/) [in](https://github.com/kowainik/summoner) [other](https://crates.io/crates/cargo-scaffold) [ecosystems](https://github.com/golang-standards/project-layout). Scaffolding systems are employed to ease onboarding of new developers. Within `pack` this feature is targeted at onboarding buildpack authors. + +- Explaining the feature largely in terms of examples. + +`pack buildpack new --libcnb` should generate skeleton code according to the [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: + +```bash +. +├── buildpack.toml +├── cmd +│ └── main.go +├── go.mod +├── go.sum +└── pkg + ├── build.go + └── detect.go +``` + +This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the `--libcnb` skeleton. `pack buildpack new --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with `--libcnb` onboarding and support experienced buildpack authors to use the scaffolding of their choice. + +# How it Works +[how-it-works]: #how-it-works + +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from an embedded file system. + +A full invocation to generate `libcnb` scaffolding is similar to the [documented project scaffolding](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). + +```bash +pack buildpack new --libcnb examples/ruby \ + --api 0.7 \ + --path ruby-buildpack \ + --version 0.0.1 \ + --stacks io.buildpacks.samples.stacks.bionic +``` + +A full session includes the terminal prompts that the project scaffolding tool asks of the end user: + +```bash +pack buildpack new --libcnb examples/ruby \ + --api 0.7 \ + --path ruby-buildpack \ + --version 0.0.1 \ + --stacks io.buildpacks.samples.stacks.bionic +Package name (often github.com/username/repo)? github.com/AidanDelaney/ruby +Created project in ruby-buildpack +``` + +Templates are provided as a file tree. A root-level `prompts.yaml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and golang skeleton code is: + +```bash +. +├── prompts.yaml +├── cmd + └── main.go +``` + +Where `prompts.yaml` is of the form: + +```yaml +prompts: + - variable: packageName + prompt: Package name (often github.com/username/repo) +``` + +And an example templated source file is: + +```golang +package main + +import ( + "github.com/buildpacks/libcnb" + + bp "{{ .packageName }}/pkg" +) + +func main() { + libcnb.Main( + bp.Detect{}, + bp.Build{}, + ) +} +``` + +The following template variables, known as the _reserved template variables_ are available for substitution in all project files: + +* `.ID` -- the buildpack id that will be written to `buildpack.toml` obtained from a positional argument to `pack` +* `.Version` -- the buildpack version obtained from the argument to the `--version` flag +* `.API` -- the buildpack API version obtained from the argument to the `--api` flag +* `.Stacks` -- a slice containing the buildpack stacks obtained from the argument to the `--stacks` flag + +Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file and not redefine reserved template variables. An optional default value may be provided. Variables that do not provide a default value must + +```yaml +prompts: + - variable: packageName + prompt: Package name (often github.com/username/repo) + default: DefaultValue +``` + +Answers to prompts can be provided in a yaml file. This supports programmatic use of `pack`: + +``` +pack buildpack new --libcnb examples/ruby \ + --config-file answers.yaml \ + --api 0.7 \ + --path ruby-buildpack \ + --version 0.0.1 \ + --stacks io.buildpacks.samples.stacks.bionic +Created project in ruby-buildpack +``` + +The format of `--config-file` parallels the format used in `cookiecutter`. An example of which is: + +```yaml +defaultContext: + packageName: com.example/an-example +``` + +All input files are expected to be normalized to Unix line endings. + +# Migration +[migration]: #migration + +The current `bash` project scaffolding can be embedded in `pack` as an embedded file system. This would allow the `--libcnb` and `--target URL` code to be reused. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +* Golang project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. +* Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpaks/bash my_buildpack`. +* Including `--libcnb` project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of the embedded file system plus the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). The `pack` binary will grow to +* This proposal commits to support a specific project scaffolding format. A migration path should be established if and when a de-facto standard golang template library becomes available. +* In supporting `--libcnb` we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + +We have considered a wider re-implementation of [cookiecutter](https://cookiecutter.readthedocs.io/). However, the scope of a re-implementation was considered too wide. In addition, we have considered a cookiecutter-lite implementation where project scaffolding is cloned from a repository. This proposal recommends a project scaffolding that can be embedded in `pack` and, in addition, can be downloaded from an Internet source. + +We have also considered [springerle](https://github.com/carlmjohnson/springerle) which uses a single txtar file to describe skeleton project structure. The use of a single txtar file requires template providers to write and maintain this format. + +[bare](https://github.com/bare-cli/bare) is a new project. It assumes that all templates are stored on github.com. + +- Why is this proposal the best? +To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that embeds cnb-provided skeletons in `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. + +- What is the impact of not doing this? + +Omitting support for `libcnb` project scaffolding requires new buildpack authors to consult our documentation about project structure. Moreover, as `pack` supports scaffolding of shell-script buildpacks the impression is given that the buildpacks project _prefers_ shell implementations. + +# Prior Art +[prior-art]: #prior-art + +There are many, many competing implementations of project scaffolding tools. + +* Python's [Cookiecutter](https://cookiecutter.readthedocs.io/): widely used to scaffold projects in many languages, including golang. Requires a Python runtime to be available. +* [springerle](https://github.com/carlmjohnson/springerle): golang re-think of cookiecutter. Springerle uses a single txtar file augmented with a header containing user prompts. This proposal prefers using a filesystem rather than a single txtar file as the filesystem approach extends to cloning repositories. +* [cgapp](https://github.com/create-go-app/cli): This proposal heavily borrows from cgapp. It is necessary to modify parts of cgapp in order to apply variable substitution. +* JavaScript's [Yeoman](https://yeoman.io/): widely used in the web ecosystem +* [boilr](https://github.com/tmrts/boilr/): golang cookiecutter. Moribund project. +* [bare](https://github.com/bare-cli/bare): golang cookiecutter. Possible successor to boilr. Both boilr and bare assume the templates are stored on github.com and use the zip downloading functionality of that specific service. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +This proposal does not require any spec changes. From 191b3a4706bf3034ab1fd06f4123304d8333f58d Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 25 Mar 2022 07:47:49 +0000 Subject: [PATCH 184/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Terence Lee Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 2a8be7891..2377bdc40 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -22,7 +22,7 @@ The `pack buildpack new` subcommand creates a new buildpack based on a shell-scr The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal `libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. -Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--target URL` allows other projects to re-use `pack` as their scaffolding tool. +Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--template URL` allows other projects to re-use `pack` as their scaffolding tool. `pack buildpack new --libcnb` supports end-users who wish to scaffold a new `libcnb`-based buildpack. The operation of `pack buildpack new --libcnb` does not require internet access to succeed. `pack buildpack new --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. From 65877a02d96c9033b0b1355ebc901acdfe07988b Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 6 May 2022 10:38:54 +0100 Subject: [PATCH 185/372] Replace `buildpack new` with `buildpack create` We replace the existing functionality where variables are provided to `pack buildpack new` as flags with `pack buildpack create` which prompts end-users for inputs. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 148 ++++++++++--------- 1 file changed, 77 insertions(+), 71 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 2377bdc40..957192a51 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -7,10 +7,12 @@ - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) +- Supersedes: https://github.com/buildpacks/rfcs/blob/main/text/0077-pack-buildpack-create.md # Summary -The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal extends `pack buildpack new` with a `--libcnb` option and a `--template URL` option to allow alternatives to the shell-script template. The `--libcnb` option creates a new buildpack written in golang using `libcnb`. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. +The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack prompting the end-user to choose an implementation language. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. + +We will implement the project scaffolding logic in a Go module separate from `pack`. # Definitions [definitions]: #definitions @@ -20,22 +22,18 @@ The `pack buildpack new` subcommand creates a new buildpack based on a shell-scr # Motivation [motivation]: #motivation -The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal `libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. - -Extending `pack buildpack new` with a `--libcnb` flag supports buildpack creation using the existing documented workflow. This results easier on-boarding of buildpack authors into the `libcnb` ecosystem. Additionally extending `pack buildpack new` with `--template URL` allows other projects to re-use `pack` as their scaffolding tool. +The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal Go/`libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. -`pack buildpack new --libcnb` supports end-users who wish to scaffold a new `libcnb`-based buildpack. The operation of `pack buildpack new --libcnb` does not require internet access to succeed. `pack buildpack new --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. +`pack buildpack create` supports end-users who wish to scaffold a new bash or Go/`libcnb`-based buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. -Extending `pack buildpack new` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. +Replacing `pack buildpack new` with `pack buildpack create` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. # What it is [what-it-is]: #what-it-is Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](https://yeoman.io/) [in](https://github.com/kowainik/summoner) [other](https://crates.io/crates/cargo-scaffold) [ecosystems](https://github.com/golang-standards/project-layout). Scaffolding systems are employed to ease onboarding of new developers. Within `pack` this feature is targeted at onboarding buildpack authors. -- Explaining the feature largely in terms of examples. - -`pack buildpack new --libcnb` should generate skeleton code according to the [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: +`pack buildpack create` should prompt the end user to choose an implementation language. Thereafter, generate skeleton code according to the best practices for that language. For example, users choosing Go as an implementation language would generate a [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: ```bash . @@ -49,50 +47,79 @@ Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](ht └── detect.go ``` -This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the `--libcnb` skeleton. `pack buildpack new --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with `--libcnb` onboarding and support experienced buildpack authors to use the scaffolding of their choice. +This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the generated project. `pack buildpack create --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with onboarding and support experienced buildpack authors to use the scaffolding of their choice. # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from an embedded file system. +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. -A full invocation to generate `libcnb` scaffolding is similar to the [documented project scaffolding](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). +A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). ```bash -pack buildpack new --libcnb examples/ruby \ - --api 0.7 \ - --path ruby-buildpack \ - --version 0.0.1 \ - --stacks io.buildpacks.samples.stacks.bionic +pack buildpack new +Use the arrow keys to navigate: ↓ ↑ → ← +? Choose a project template: + ▸ bash + go + ``` A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```bash -pack buildpack new --libcnb examples/ruby \ - --api 0.7 \ - --path ruby-buildpack \ - --version 0.0.1 \ - --stacks io.buildpacks.samples.stacks.bionic -Package name (often github.com/username/repo)? github.com/AidanDelaney/ruby -Created project in ruby-buildpack +pack buildpack new +Use the arrow keys to navigate: ↓ ↑ → ← +? Choose a project template: + ▸ bash + go +✔ Enter a directory in which to scaffold the project: bash_buildpack +Use the arrow keys to navigate: ↓ ↑ → ← +? Choose the buildpack API version (use the default if you are unsure): + ▸ 0.7 + 0.8 +✔ Enter an identifier for the buildpack: example/bash +✔ Enter the initial buildpack version: 0.0.1 +✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic + +Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.yaml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and golang skeleton code is: +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and Go skeleton code is: ```bash . -├── prompts.yaml -├── cmd - └── main.go +├── prompts.toml +└── {{.ProjectDirectory}} + ├── buildpack.toml + ├── cmd + │   └── main.go + ├── go.mod + └── pkg + ├── build.go + └── detect.go ``` -Where `prompts.yaml` is of the form: +Where `prompts.toml` is of the form: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +default="bash_buildpack" +required=true + +[[prompt]] +name="BuildpackApi" +prompt="Choose the buildpack API version (use the default if you are unsure)" +choices=["0.7", "0.8"] -```yaml -prompts: - - variable: packageName - prompt: Package name (often github.com/username/repo) +[[prompt]] +name="ModuleName" +prompt="Enter a valid Go module name for this buildpack" +default="github.com/user/buildpack" + +... ``` And an example templated source file is: @@ -103,7 +130,7 @@ package main import ( "github.com/buildpacks/libcnb" - bp "{{ .packageName }}/pkg" + bp "{{ .ModuleName }}/pkg" ) func main() { @@ -114,39 +141,20 @@ func main() { } ``` -The following template variables, known as the _reserved template variables_ are available for substitution in all project files: - -* `.ID` -- the buildpack id that will be written to `buildpack.toml` obtained from a positional argument to `pack` -* `.Version` -- the buildpack version obtained from the argument to the `--version` flag -* `.API` -- the buildpack API version obtained from the argument to the `--api` flag -* `.Stacks` -- a slice containing the buildpack stacks obtained from the argument to the `--stacks` flag - -Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file and not redefine reserved template variables. An optional default value may be provided. Variables that do not provide a default value must +Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. -```yaml -prompts: - - variable: packageName - prompt: Package name (often github.com/username/repo) - default: DefaultValue -``` - -Answers to prompts can be provided in a yaml file. This supports programmatic use of `pack`: +Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`: ``` -pack buildpack new --libcnb examples/ruby \ - --config-file answers.yaml \ - --api 0.7 \ - --path ruby-buildpack \ - --version 0.0.1 \ - --stacks io.buildpacks.samples.stacks.bionic -Created project in ruby-buildpack +pack buildpack create --overrides overrides.toml +Created project in bash_buildpack ``` -The format of `--config-file` parallels the format used in `cookiecutter`. An example of which is: +The format of `--overrides` parallels the `prompts.toml` format. An example of which is: -```yaml -defaultContext: - packageName: com.example/an-example +```toml +ProjectDirectory="test_example" +ModuleName="github.com/test/test" ``` All input files are expected to be normalized to Unix line endings. @@ -154,7 +162,7 @@ All input files are expected to be normalized to Unix line endings. # Migration [migration]: #migration -The current `bash` project scaffolding can be embedded in `pack` as an embedded file system. This would allow the `--libcnb` and `--target URL` code to be reused. +The current `bash` project scaffolding can be re-used in the default project scaffolding. # Drawbacks [drawbacks]: #drawbacks @@ -162,28 +170,26 @@ The current `bash` project scaffolding can be embedded in `pack` as an embedded Why should we *not* do this? * Golang project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. -* Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpaks/bash my_buildpack`. -* Including `--libcnb` project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of the embedded file system plus the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). The `pack` binary will grow to +* Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpacks/bash my_buildpack`. +* Including generalized project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). * This proposal commits to support a specific project scaffolding format. A migration path should be established if and when a de-facto standard golang template library becomes available. -* In supporting `--libcnb` we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. +* In default project scaffolding we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. # Alternatives [alternatives]: #alternatives - What other designs have been considered? -We have considered a wider re-implementation of [cookiecutter](https://cookiecutter.readthedocs.io/). However, the scope of a re-implementation was considered too wide. In addition, we have considered a cookiecutter-lite implementation where project scaffolding is cloned from a repository. This proposal recommends a project scaffolding that can be embedded in `pack` and, in addition, can be downloaded from an Internet source. - We have also considered [springerle](https://github.com/carlmjohnson/springerle) which uses a single txtar file to describe skeleton project structure. The use of a single txtar file requires template providers to write and maintain this format. [bare](https://github.com/bare-cli/bare) is a new project. It assumes that all templates are stored on github.com. - Why is this proposal the best? -To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that embeds cnb-provided skeletons in `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. +To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that integrates cnb-provided skeletons with `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. - What is the impact of not doing this? -Omitting support for `libcnb` project scaffolding requires new buildpack authors to consult our documentation about project structure. Moreover, as `pack` supports scaffolding of shell-script buildpacks the impression is given that the buildpacks project _prefers_ shell implementations. +Omitting support for generalized project scaffolding requires new buildpack authors to consult our documentation about project structure. Moreover, as `pack` supports scaffolding of shell-script buildpacks the impression is given that the buildpacks project _prefers_ shell implementations. # Prior Art [prior-art]: #prior-art @@ -192,7 +198,7 @@ There are many, many competing implementations of project scaffolding tools. * Python's [Cookiecutter](https://cookiecutter.readthedocs.io/): widely used to scaffold projects in many languages, including golang. Requires a Python runtime to be available. * [springerle](https://github.com/carlmjohnson/springerle): golang re-think of cookiecutter. Springerle uses a single txtar file augmented with a header containing user prompts. This proposal prefers using a filesystem rather than a single txtar file as the filesystem approach extends to cloning repositories. -* [cgapp](https://github.com/create-go-app/cli): This proposal heavily borrows from cgapp. It is necessary to modify parts of cgapp in order to apply variable substitution. +* [cgapp](https://github.com/create-go-app/cli): This proposal heavily borrows from cgapp. * JavaScript's [Yeoman](https://yeoman.io/): widely used in the web ecosystem * [boilr](https://github.com/tmrts/boilr/): golang cookiecutter. Moribund project. * [bare](https://github.com/bare-cli/bare): golang cookiecutter. Possible successor to boilr. Both boilr and bare assume the templates are stored on github.com and use the zip downloading functionality of that specific service. From 5175271082212c38638ec5d7f74f3ae0d0a5d8d4 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 10 Jun 2022 06:36:24 +0100 Subject: [PATCH 186/372] Pass pack buildpack create overrides as CLI parameters Update the RFC so that overrides to buildpack creation variables can be passed on the command line. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 957192a51..19203a850 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -52,7 +52,7 @@ This approach is not opinionated on topics such as which testing framework to us # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We explicitly do not want to re-implement cookiecutter in golang due to the scope of such an implementation. Nor do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). @@ -146,17 +146,10 @@ Template variables introduced in `prompts.yaml` are required to have a name uniq Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`: ``` -pack buildpack create --overrides overrides.toml +pack buildpack create --override 'ProjectDirectory="test_example"' --overrride='ModuleName="github.com/test/test"' Created project in bash_buildpack ``` -The format of `--overrides` parallels the `prompts.toml` format. An example of which is: - -```toml -ProjectDirectory="test_example" -ModuleName="github.com/test/test" -``` - All input files are expected to be normalized to Unix line endings. # Migration From 527930550b4084a5fbc8404baca7b28a6722eed9 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 10 Jun 2022 06:49:44 +0100 Subject: [PATCH 187/372] Modify list indicators Consistently use '-' to denote a list item Co-authored-by: Javier Romero Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 19203a850..628fae5d7 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -199,9 +199,6 @@ There are many, many competing implementations of project scaffolding tools. # Unresolved Questions [unresolved-questions]: #unresolved-questions -- What parts of the design do you expect to be resolved before this gets merged? -- What parts of the design do you expect to be resolved through implementation of the feature? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From 83589056a08db1485455e96cdb98c6c7168a7f02 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 14 Jun 2022 08:51:43 +0100 Subject: [PATCH 188/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Daniel Mikusa Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 628fae5d7..cbb2d2fc6 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -52,7 +52,7 @@ This approach is not opinionated on topics such as which testing framework to us # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). From b101b59d729a247f445cdaaa3f3c8cd4ec2b57e3 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 14 Jun 2022 08:51:52 +0100 Subject: [PATCH 189/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Daniel Mikusa Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index cbb2d2fc6..b7a3aa261 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -57,7 +57,7 @@ The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1 A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). ```bash -pack buildpack new +pack buildpack create Use the arrow keys to navigate: ↓ ↑ → ← ? Choose a project template: ▸ bash From b5d3fe9993bdbd0d1415ec26c7369e608af7ce7e Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 14 Jun 2022 08:51:59 +0100 Subject: [PATCH 190/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Daniel Mikusa Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index b7a3aa261..3ceccc39b 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -68,7 +68,7 @@ Use the arrow keys to navigate: ↓ ↑ → ← A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```bash -pack buildpack new +pack buildpack create Use the arrow keys to navigate: ↓ ↑ → ← ? Choose a project template: ▸ bash From ddb10163bd0ecaf27168c91f42062696205537a1 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Wed, 29 Jun 2022 09:55:39 +0100 Subject: [PATCH 191/372] Update text/0000-pack-buildpack-new-alternatives.md Co-authored-by: Javier Romero Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 3ceccc39b..3eab7b956 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -198,7 +198,10 @@ There are many, many competing implementations of project scaffolding tools. # Unresolved Questions [unresolved-questions]: #unresolved-questions - +* Where do the template repositories live for `bash` and `golang`? + * +* What team maintains template repositories? + * Buildpack Author Tooling team # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From 2b18288bb801aa0bbaa1b51fbbc86ed16701595c Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Wed, 29 Jun 2022 09:51:59 +0100 Subject: [PATCH 192/372] Reduce the proposal scope The proposal scope is reduced to providing a more generic mechanism to scaffold buildpacks written in bash. The provision of libcnb project scaffolding will be the focus of a subsequent RFC. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 95 +++++--------------- 1 file changed, 24 insertions(+), 71 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 3eab7b956..8b87c8890 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -10,7 +10,7 @@ - Supersedes: https://github.com/buildpacks/rfcs/blob/main/text/0077-pack-buildpack-create.md # Summary -The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack prompting the end-user to choose an implementation language. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. +The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack using the existing `bash` scaffolding. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. We will implement the project scaffolding logic in a Go module separate from `pack`. @@ -22,9 +22,9 @@ We will implement the project scaffolding logic in a Go module separate from `pa # Motivation [motivation]: #motivation -The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a minimal Go/`libcnb` project template to `pack` and allows 3d parties to provide templates to create buildpacks for their chosen technology stack. +The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a mechanism to `pack` that would allow the buildpacks project to provide `libcnb` buildpack project scaffolding. -`pack buildpack create` supports end-users who wish to scaffold a new bash or Go/`libcnb`-based buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. +`pack buildpack create` supports end-users who wish to scaffold a new bash buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. Replacing `pack buildpack new` with `pack buildpack create` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. @@ -33,71 +33,38 @@ Replacing `pack buildpack new` with `pack buildpack create` allows buildpack aut Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](https://yeoman.io/) [in](https://github.com/kowainik/summoner) [other](https://crates.io/crates/cargo-scaffold) [ecosystems](https://github.com/golang-standards/project-layout). Scaffolding systems are employed to ease onboarding of new developers. Within `pack` this feature is targeted at onboarding buildpack authors. -`pack buildpack create` should prompt the end user to choose an implementation language. Thereafter, generate skeleton code according to the best practices for that language. For example, users choosing Go as an implementation language would generate a [golang standard project layout](https://github.com/golang-standards/project-layout) and include `libcnb` best practices. The layout would look similar to the following: - -```bash -. -├── buildpack.toml -├── cmd -│ └── main.go -├── go.mod -├── go.sum -└── pkg - ├── build.go - └── detect.go -``` - -This approach is not opinionated on topics such as which testing framework to use. Only components under github.com/buildpacks/* are used in the generated project. `pack buildpack create --template URL` allows for more opinionated project scaffolding. As such, we support new buildpack authors with onboarding and support experienced buildpack authors to use the scaffolding of their choice. +`pack buildpack create` creates the same buildpack scaffolding as `pack buildpack new`. However, `pack buildpack create --template URL` can be used to access alternative project scaffolding. # How it Works [how-it-works]: #how-it-works -The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not we want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell and libcnb skeletons from a cloned git repoistory. - -A full invocation to generate `bash` scaffolding prompts for the values [currently documented as flags](https://buildpacks.io/docs/buildpack-author-guide/create-buildpack/building-blocks-cnb/#using-the-pack-cli). +The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell scaffolding from a cloned git repoistory. -```bash -pack buildpack create -Use the arrow keys to navigate: ↓ ↑ → ← -? Choose a project template: - ▸ bash - go - -``` +The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--stacks` command line flags. These command line flags will be preserved in `pack buildpack create`. However, where a flag is omitted, the end-user will be prompted to provide an appropriate value. A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```bash -pack buildpack create -Use the arrow keys to navigate: ↓ ↑ → ← -? Choose a project template: - ▸ bash - go +pack buildpack create example/bash ✔ Enter a directory in which to scaffold the project: bash_buildpack Use the arrow keys to navigate: ↓ ↑ → ← ? Choose the buildpack API version (use the default if you are unsure): ▸ 0.7 0.8 -✔ Enter an identifier for the buildpack: example/bash -✔ Enter the initial buildpack version: 0.0.1 ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example, a partial file tree containing user prompts and Go skeleton code is: +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as.: ```bash . ├── prompts.toml └── {{.ProjectDirectory}} - ├── buildpack.toml - ├── cmd - │   └── main.go - ├── go.mod - └── pkg - ├── build.go - └── detect.go + └── bin + ├── build + └── detect ``` Where `prompts.toml` is of the form: @@ -114,39 +81,24 @@ name="BuildpackApi" prompt="Choose the buildpack API version (use the default if you are unsure)" choices=["0.7", "0.8"] -[[prompt]] -name="ModuleName" -prompt="Enter a valid Go module name for this buildpack" -default="github.com/user/buildpack" - ... ``` -And an example templated source file is: - -```golang -package main - -import ( - "github.com/buildpacks/libcnb" +Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. - bp "{{ .ModuleName }}/pkg" -) +Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. -func main() { - libcnb.Main( - bp.Detect{}, - bp.Build{}, - ) -} +``` +pack buildpack create --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' +✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic +Created project in bash_buildpack ``` -Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. - -Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`: +The above overrides are equivalent to: ``` -pack buildpack create --override 'ProjectDirectory="test_example"' --overrride='ModuleName="github.com/test/test"' +pack buildpack create --path test_example --api 0.8 +✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` @@ -157,16 +109,17 @@ All input files are expected to be normalized to Unix line endings. The current `bash` project scaffolding can be re-used in the default project scaffolding. +We intend to maintain `pack buildpack new` in parallel with `pack buildpack create` for two `pack` releases. After two `pack` releases, `pack buildpack new` will be replaced with an error instructing the user to run `pack buildpack create`. We will remove `pack buildpack new` after three `pack` releases. + # Drawbacks [drawbacks]: #drawbacks Why should we *not* do this? -* Golang project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. +* Project structure is straightforward and it could be better to document a `libcnb` project structure. However, given that we already support `pack builpack new`, we feel it important that `pack` also creates generation of `libcnb`-style projects. * Project scaffolding could be delegated to a 3rd party tool. The current `pack buildpack new` functionality could be extracted from `pack` and, instead, we could suggest another tool for end users to use for project scaffolding eg: `bare use buildpacks/bash my_buildpack`. * Including generalized project scaffolding in `pack` will increase size of `pack` binary. The size of `pack` will increase by the size of an appropriate prompting package (such as [survey](https://pkg.go.dev/github.com/AlecAivazis/survey/v2)) and an appropriate package allowing `git clone` (such as [go-git](https://github.com/go-git/go-git) at ). * This proposal commits to support a specific project scaffolding format. A migration path should be established if and when a de-facto standard golang template library becomes available. -* In default project scaffolding we cannot be opinionated about test frameworks; this will result in scaffolded projects with no default test setup. # Alternatives [alternatives]: #alternatives @@ -178,7 +131,7 @@ We have also considered [springerle](https://github.com/carlmjohnson/springerle) [bare](https://github.com/bare-cli/bare) is a new project. It assumes that all templates are stored on github.com. - Why is this proposal the best? -To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that integrates cnb-provided skeletons with `pack` and allows `pack` to clone 3rd party skeletons from a location chosen by the end-user. +To integrate with `pack` we want a pure-golang project scaffolding tool. This proposal advocates an approach that integrates future cnb-provided project scaffolding with `pack` and allows `pack` to clone 3rd party project scaffolding from a location chosen by the end-user. - What is the impact of not doing this? From 2f014c7934da79435cf474366f2adc110c098ef0 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Wed, 29 Jun 2022 10:03:21 +0100 Subject: [PATCH 193/372] Add buildpack id to examples Provide more complete CLI examples Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 8b87c8890..09f3899c1 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -89,7 +89,7 @@ Template variables introduced in `prompts.yaml` are required to have a name uniq Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. ``` -pack buildpack create --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' +pack buildpack create example/bash --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` @@ -97,7 +97,7 @@ Created project in bash_buildpack The above overrides are equivalent to: ``` -pack buildpack create --path test_example --api 0.8 +pack buildpack create example/bash --path test_example --api 0.8 ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` From d3747adfc5849f4a8a703d5b46fc394699b071bc Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Thu, 14 Jul 2022 09:13:46 +0100 Subject: [PATCH 194/372] Provide fuller specification of prompt.toml Better specify prompts.toml and introduce both templates and template collections by example. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 172 +++++++++++++++++-- 1 file changed, 159 insertions(+), 13 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 09f3899c1..17b3ee142 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -12,7 +12,7 @@ # Summary The `pack buildpack new` subcommand creates a new buildpack based on a shell-script template. This proposal replaces `pack buildpack new` with `pack buildpack create` to allow alternatives to the shell-script template. Invoking `pack buildpack create` creates a new buildpack using the existing `bash` scaffolding. The `--template URL` option allows buildpack creation from an arbitrary 3rd party URL. -We will implement the project scaffolding logic in a Go module separate from `pack`. +We will implement the project scaffolding logic in a Go module separate from `pack`. This decouples `pack` from the project scaffolding implementation. # Definitions [definitions]: #definitions @@ -24,7 +24,7 @@ We will implement the project scaffolding logic in a Go module separate from `pa The creation of buildpacks in languages other than `bash` is undocumented in the [Buildpack Author Guide](https://buildpacks.io/docs/buildpack-author-guide). In particular, creating a buildpack in Go using `libcnb` is undocumented because there is no mechanism to generate a scaffolded project. This proposal adds a mechanism to `pack` that would allow the buildpacks project to provide `libcnb` buildpack project scaffolding. -`pack buildpack create` supports end-users who wish to scaffold a new bash buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. +`pack buildpack create` supports end-users who wish to scaffold a new buildpack. The operation of `pack buildpack create` requires Internet access to succeed. `pack buildpack create --template URL` allows projects (such as Paketo, Heroku and others) to define skeleton projects for new buildpacks. Replacing `pack buildpack new` with `pack buildpack create` allows buildpack authors to more-easily create new buildpacks in their chosen language and framework. @@ -44,8 +44,8 @@ The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--st A full session includes the terminal prompts that the project scaffolding tool asks of the end user: -```bash -pack buildpack create example/bash +```command +$ pack buildpack create example/bash ✔ Enter a directory in which to scaffold the project: bash_buildpack Use the arrow keys to navigate: ↓ ↑ → ← ? Choose the buildpack API version (use the default if you are unsure): @@ -56,7 +56,16 @@ Use the arrow keys to navigate: ↓ ↑ → ← Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as.: +The user can skip common prompts by providing `--api`, `--path` or `--stacks` as command line flags. Running the same example as above, but providing both `--api` and `--stacks`, but not `--path`, results in the user being prompted only to provide the output directory. We provide more detail on how flags override prompts in a subsection below. + +```command +$ pack buildpack create example/bash --api 0.8 --stacks io.buildpacks.samples.stacks.bionic +✔ Enter a directory in which to scaffold the project: bash_buildpack + +Created project in bash_buildpack +``` + +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as follows, a full specification of `prompts.toml` is provided in the following subsection. ```bash . @@ -84,25 +93,162 @@ choices=["0.7", "0.8"] ... ``` -Template variables introduced in `prompts.yaml` are required to have a name unique within the `prompts.yaml` file. An optional default value may be provided. - -Answers to prompts can be provided in a toml file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. +Answers to prompts can be provided in an `overrides.toml` file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. -``` -pack buildpack create example/bash --override 'ProjectDirectory="test_example"' --override='BuildpackApi=0.8' +```command +$ pack buildpack create example/bash --override='ProjectDirectory="test_example"' --override='BuildpackApi=0.8' ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` The above overrides are equivalent to: -``` -pack buildpack create example/bash --path test_example --api 0.8 +```command +$ pack buildpack create example/bash --path test_example --api 0.8 ✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` -All input files are expected to be normalized to Unix line endings. +Multiple templates can be managed in a single repository, referred to as a **template colleciton**. + +## Template Collections + +We expect project templates to be managed in `git` repositories. Projects may wish to manage multiple templates in a single repository. We refer to this as a template collection. More formally, a template collection is a `git` repository where top-level directories are templates. A template collection does not contain a top-level `prompts.toml`, but top-level subdirectories are expected to contain `prompts.toml` files. As an example, a template collection might have the following structure where the `bash` sub-directory is a template and the `Go` sub-directory is a template. + +```bash +. +├── bash +└── Go +``` + +A user may use a template collection as an argument to `--template`. A template collection passed as `--template` first prompts the end user to choose between the available templates in the repository. Once a template is chosen, the user is prompted using the `prompts.toml` from the chosen template. + +```command +$ pack buildpack create --template https://github.com/AidanDelaney/cnb-buildpack-templates +Use the arrow keys to navigate: ↓ ↑ → ← +? choose a project template: + ▸ Go + bash +Enter a directory in which to scaffold the project: go_buildpack +? Choose the buildpack API version (use the default if you are unsure): + 0.7 + ▸ 0.8 +✔ 0.8 +Enter an identifier for the buildpack: example/golang +Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic +✔ Enter a valid Go module name for this buildpack: github.com/user/buildpack +Created project in go_buildpack +``` + +A user may specify the choice of template from a template collection via a `--sub-path` flag. A user may choose a `bash` template from a template collection and provide the api, output directory and stacks as command line flags: + +```command +$ pack buildpack create example/golang --template https://github.com/AidanDelaney/cnb-buildpack-templates \ + --sub-path Go \ + --api 0.8 \ + --path go_buildpack \ + --stacks io.buildpacks.samples.stacks.bionic +Created project in go_buildpack +``` + +Having provided examples of the use of `pack buildpack create`. We now specify the format of `prompts.toml`. + +## prompts.toml + +The prompts are contained in a TOML file. The `prompts.toml` file contains an array of tables. Each table specifies a single `prompt`. Each prompt defines a variable identified by a provided `name`. + +| field | required | description | type | +|---------------|----------|------------------------------------------------------------------------------------------------------------------------------------------|----------------| +| name | Yes | The variable identifier. Must be unique within the `prompts.toml` file. | string | +| prompt | Yes | The default prompt question to ask the user | string | +| prompt.locale | No | A translation of the default prompt into the locale specified. | | +| required | No | Evaluates to `false` if not provided. Specifies whether the user **must** provide a value for the variable identified by `name`. | boolean | +| default | No | A default value for the variable identified by `name`. Mutually exclusive to `choices` | string | +| choices | No | A list of default values from which the user may choose a value for the variable identified by `name`. Mutually exclusive to `default`. | list of stings | + +Prompt locales are specified following [IETF BCP 47](https://tools.ietf.org/rfc/bcp/bcp47.txt). As such `prompt.en-US` is the translation of `prompt` into US English and `prompt.es-MX` is the translation of `prompt` into Mexican Spanish. + +For example, the following is a valid `prompts.toml` file: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +``` + +The following `prompts.toml` is invalid as it does not contain the required `name` field: + +```toml +[[prompt]] +prompt="Enter a directory in which to scaffold the project" +``` + +The following `prompts.toml` is invalid as value of the `name` field is not unique within the document: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" + +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory" +``` + +The following `prompts.toml` is invalid as the `default` and `choices` fields are mutually excusive: + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +default="/tmp" +choices=["/tmp", "/home"] +``` + +### Overrides + +If a template contains both a `prompts.toml` file and an `overrides.toml` file, then the `overrides.toml` file provides values for variables introduced in `prompts.toml`. + +Given a `prompts.toml` file + +```toml +[[prompt]] +name="ProjectDirectory" +prompt="Enter a directory in which to scaffold the project" +``` + +And an `overrides.toml` file + +```toml +ProjectDirectory="/tmp" +``` + +Then the value "/tmp" is interpreted as the value of the variable identified by `ProjectDirectory`. + + +Overrides may also be provided as command line arguments. For example `pack buildpack create --overrides ProjectDirectory=/tmp` provides a value, `/tmp`, for the variable identified by the name `ProjectDirectory`. + +Overrides provided as command line flags have **higher precedence** than overrides provided in `overrides.toml`. + +When an override is provided `pack buildpack create` must not prompt the user for overriden values. + +## `pack buildpack create` flags as prompts variables + +If specified, the following `pack buildpack create` flags override the value of variables with the listed identifier: + +| command line flag | variable name | +|-------------------|--------------------| +| `--path` | `ProjectDirectory` | +| `--api` | `BuildpackApi` | +| `--stacks` | `BuildpackStacks` | + +In addition, the required positional argument to `pack buildpack create`, eg: `pack buildpack create example/bash`, overrides the variable identified by `BuildpackID`. + +## `pack buildpack create` Substitutions + +The operation of variable substitution follows the operation of Go [`text/template`](https://pkg.go.dev/text/template). Prompts defined in `prompts.toml` are interpreted, the user may be prompted and set of variables is generated. The identifiers of variables are replaced with the value of the variable. + +Variables may be used in files, for example where a `prompts.toml` defines a variable with identifier `ProjectDirectory` then the expression `{{.ProjectDirectory}}` in files is replaced with the value of the variable identified by `ProjectDirectory`. In addition, the expression `{{.ProjectDirectory}}` may be used in template file paths. The generated project substitutes file paths with variable expressions with the value of the variable. # Migration [migration]: #migration From 926498087bbc1a1165be6111f5c39e0f12d9ca1c Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Thu, 1 Sep 2022 07:38:36 +0100 Subject: [PATCH 195/372] Remove overrides concept End-users can provide arguments as key/value pairs. The key must match the name of a prompt in prompts.toml. --- text/0000-pack-buildpack-new-alternatives.md | 85 ++++++-------------- 1 file changed, 26 insertions(+), 59 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 17b3ee142..4a62a04d9 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -40,12 +40,13 @@ Project scaffolding is [very](https://cookiecutter.readthedocs.io/) [popular](ht The design is modeled on [cookiecutter](https://cookiecutter.readthedocs.io/en/1.7.2/) with reference to [springerle](https://github.com/carlmjohnson/springerle) -- a similar implementation in golang -- and [create-go-app](https://github.com/create-go-app/). We do not want to `os.Exec` a python subprocess to run cookiecutter as this would require availability of a python runtime environment. Instead we propose to borrow heavily from create-go-app, and generate the default shell scaffolding from a cloned git repoistory. -The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--stacks` command line flags. These command line flags will be preserved in `pack buildpack create`. However, where a flag is omitted, the end-user will be prompted to provide an appropriate value. +The `pack buildpack new` project scaffolding accepts `--api`, `--path` and `--stacks` command line flags and the buildpack id as a positional argument. These command line flags and positional arguments will be replaced with user prompts `pack buildpack create`. A full session includes the terminal prompts that the project scaffolding tool asks of the end user: ```command -$ pack buildpack create example/bash +$ pack buildpack create +✔ Enter an ID for this buildpack: example/bash ✔ Enter a directory in which to scaffold the project: bash_buildpack Use the arrow keys to navigate: ↓ ↑ → ← ? Choose the buildpack API version (use the default if you are unsure): @@ -56,16 +57,15 @@ Use the arrow keys to navigate: ↓ ↑ → ← Created project in bash_buildpack ``` -The user can skip common prompts by providing `--api`, `--path` or `--stacks` as command line flags. Running the same example as above, but providing both `--api` and `--stacks`, but not `--path`, results in the user being prompted only to provide the output directory. We provide more detail on how flags override prompts in a subsection below. +The user can skip prompts by providing `--arg key=value` as command line flags. Each of the prompts from the previous interactive invocation can be skipped by providing appropriate pairs of key and value. ```command -$ pack buildpack create example/bash --api 0.8 --stacks io.buildpacks.samples.stacks.bionic -✔ Enter a directory in which to scaffold the project: bash_buildpack +$ pack buildpack create --arg ProjectDirectory=bash_buildpack --arg BuildpackApi=0.8 --arg BuildpackID=example/bash --arg BuildpakStacks=io.buildpacks.samples.stacks.bionic Created project in bash_buildpack ``` -Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding would be structured as follows, a full specification of `prompts.toml` is provided in the following subsection. +Templates are provided as a file tree. A root-level `prompts.toml` file contains prompts for the end-user. For example the current `bash` scaffolding could be structured as follows, a full specification of `prompts.toml` is provided in the following subsection. ```bash . @@ -93,21 +93,7 @@ choices=["0.7", "0.8"] ... ``` -Answers to prompts can be provided in an `overrides.toml` file. This supports programmatic use of `pack`. Alternatively, variables specified in `prompts.toml` may have values provided as command line flags. When an override is provided in the CLI or `overrides.toml` file, then the end-user is not prompted to provide the value. - -```command -$ pack buildpack create example/bash --override='ProjectDirectory="test_example"' --override='BuildpackApi=0.8' -✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic -Created project in bash_buildpack -``` - -The above overrides are equivalent to: - -```command -$ pack buildpack create example/bash --path test_example --api 0.8 -✔ Enter a stack this buildpack will support by default: io.buildpacks.samples.stacks.bionic -Created project in bash_buildpack -``` +A specification of `prompts.toml` is provided below. Multiple templates can be managed in a single repository, referred to as a **template colleciton**. @@ -143,11 +129,12 @@ Created project in go_buildpack A user may specify the choice of template from a template collection via a `--sub-path` flag. A user may choose a `bash` template from a template collection and provide the api, output directory and stacks as command line flags: ```command -$ pack buildpack create example/golang --template https://github.com/AidanDelaney/cnb-buildpack-templates \ +$ pack buildpack create --template https://github.com/AidanDelaney/cnb-buildpack-templates \ --sub-path Go \ - --api 0.8 \ - --path go_buildpack \ - --stacks io.buildpacks.samples.stacks.bionic + --arg BuildpackID=example/golang \ + --arg BuildpackAPI=0.8 \ + --arg ProjectDirectory=go_buildpack \ + --arg BuildpackStacks=io.buildpacks.samples.stacks.bionic Created project in go_buildpack ``` @@ -195,7 +182,7 @@ name="ProjectDirectory" prompt="Enter a directory" ``` -The following `prompts.toml` is invalid as the `default` and `choices` fields are mutually excusive: +The following `prompts.toml` is invalid as the `default` and `choices` fields are mutually exclusive: ```toml [[prompt]] @@ -205,44 +192,24 @@ default="/tmp" choices=["/tmp", "/home"] ``` -### Overrides +## Arguments -If a template contains both a `prompts.toml` file and an `overrides.toml` file, then the `overrides.toml` file provides values for variables introduced in `prompts.toml`. +Arguments may be provided as command line arguments. For example `pack buildpack create --arg ProjectDirectory=/tmp` provides a value, `/tmp`, for the variable identified by the name `ProjectDirectory`. -Given a `prompts.toml` file +When an argument, `key=value`, is provided then `pack buildpack create` must not prompt the user with the prompt identified by `name=key`. -```toml -[[prompt]] -name="ProjectDirectory" -prompt="Enter a directory in which to scaffold the project" -``` +Where an `--arg key=value` is provided and no prompt identified by `name=key` exists in `prompts.toml`, then the provided argument is ignored. -And an `overrides.toml` file +Choices constrain values for a given variable. Where a provided `value` does not match one of the `choices` then an error is returned. That is to say, given -```toml -ProjectDirectory="/tmp" +``` +[[prompt]] +name="BuildpackApi" +prompt="Choose the buildpack API version (use the default if you are unsure)" +choices=["0.7", "0.8"] ``` -Then the value "/tmp" is interpreted as the value of the variable identified by `ProjectDirectory`. - - -Overrides may also be provided as command line arguments. For example `pack buildpack create --overrides ProjectDirectory=/tmp` provides a value, `/tmp`, for the variable identified by the name `ProjectDirectory`. - -Overrides provided as command line flags have **higher precedence** than overrides provided in `overrides.toml`. - -When an override is provided `pack buildpack create` must not prompt the user for overriden values. - -## `pack buildpack create` flags as prompts variables - -If specified, the following `pack buildpack create` flags override the value of variables with the listed identifier: - -| command line flag | variable name | -|-------------------|--------------------| -| `--path` | `ProjectDirectory` | -| `--api` | `BuildpackApi` | -| `--stacks` | `BuildpackStacks` | - -In addition, the required positional argument to `pack buildpack create`, eg: `pack buildpack create example/bash`, overrides the variable identified by `BuildpackID`. +and given a CLI invocation `pack buildpack create --arg BuildpackApi=0.6` the choices are restricted to `["0.7", "0.8"]`. Therefore, `0.6` is an invalid argument and an error is returned to the end-user. ## `pack buildpack create` Substitutions @@ -250,6 +217,8 @@ The operation of variable substitution follows the operation of Go [`text/templa Variables may be used in files, for example where a `prompts.toml` defines a variable with identifier `ProjectDirectory` then the expression `{{.ProjectDirectory}}` in files is replaced with the value of the variable identified by `ProjectDirectory`. In addition, the expression `{{.ProjectDirectory}}` may be used in template file paths. The generated project substitutes file paths with variable expressions with the value of the variable. +When a source file contains a Go `text/template` style expression and the variable name does not appear in `prompts.toml`, then the `text/template` style expression is not replaced. For example, if a `README.md` file contains the expression `{{.Example}}` and `Example` is not a variable defined in `prompts.toml`, then the string `{{.Example}}` is not replaced in `README.md`. + # Migration [migration]: #migration @@ -297,8 +266,6 @@ There are many, many competing implementations of project scaffolding tools. # Unresolved Questions [unresolved-questions]: #unresolved-questions -* Where do the template repositories live for `bash` and `golang`? - * * What team maintains template repositories? * Buildpack Author Tooling team From 93e7f4222ec20704aa111a17a1f9bc9084cd0f5f Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 6 Sep 2022 05:52:05 +0100 Subject: [PATCH 196/372] Drop prompt locale Incorporating the locale introduces too much complexity in the initial RFC. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 4a62a04d9..201c3650c 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -148,13 +148,10 @@ The prompts are contained in a TOML file. The `prompts.toml` file contains an a |---------------|----------|------------------------------------------------------------------------------------------------------------------------------------------|----------------| | name | Yes | The variable identifier. Must be unique within the `prompts.toml` file. | string | | prompt | Yes | The default prompt question to ask the user | string | -| prompt.locale | No | A translation of the default prompt into the locale specified. | | | required | No | Evaluates to `false` if not provided. Specifies whether the user **must** provide a value for the variable identified by `name`. | boolean | | default | No | A default value for the variable identified by `name`. Mutually exclusive to `choices` | string | | choices | No | A list of default values from which the user may choose a value for the variable identified by `name`. Mutually exclusive to `default`. | list of stings | -Prompt locales are specified following [IETF BCP 47](https://tools.ietf.org/rfc/bcp/bcp47.txt). As such `prompt.en-US` is the translation of `prompt` into US English and `prompt.es-MX` is the translation of `prompt` into Mexican Spanish. - For example, the following is a valid `prompts.toml` file: ```toml From e6052cf12f14f2daa8d5fa48e153dcb821f67dc2 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 7 Oct 2022 13:50:17 -0400 Subject: [PATCH 197/372] Query template arguments Provide `pack buildpack create --help` to query the arguments that exist in a template. Signed-off-by: Aidan Delaney --- text/0000-pack-buildpack-new-alternatives.md | 29 ++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0000-pack-buildpack-new-alternatives.md index 201c3650c..b82fbdca1 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0000-pack-buildpack-new-alternatives.md @@ -208,6 +208,35 @@ choices=["0.7", "0.8"] and given a CLI invocation `pack buildpack create --arg BuildpackApi=0.6` the choices are restricted to `["0.7", "0.8"]`. Therefore, `0.6` is an invalid argument and an error is returned to the end-user. +Arguments for buildpack creation need to be discoverable. We intend to support this via the `--help` flag to `pack buildpack create`. Executing + +``` +pack buildpack create --help + +pack buildpack create +arguments offered by template + ProjectName (default: pyexample) + BuildpackApi 0.7, 0.8 (default 0.7) +``` + +shows a default help message. Invoking `--help` with a template url queries the prompts provided by the template: + +``` +pack buildpack create --help --template= +arguments offered by template + ProjectName (default: pyexample) + PythonVersion=python3.10, python3.9, python3.8 (default: python3.10) + NumDigits (default: 3) +``` +The possible values of a template collection can similarly be queried: + +``` +pack buildpack create --help --template= +templates available in collection + Go + bash +``` + ## `pack buildpack create` Substitutions The operation of variable substitution follows the operation of Go [`text/template`](https://pkg.go.dev/text/template). Prompts defined in `prompts.toml` are interpreted, the user may be prompted and set of variables is generated. The identifiers of variables are replaced with the value of the variable. From adaa3b9fd8047b2dc23894bec67e74bf6acd162a Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Thu, 27 Oct 2022 09:01:45 +0100 Subject: [PATCH 198/372] Properly add metadata for RFC RFC --- text/{0000-rfc-process.md => 0106-rfc-process.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-rfc-process.md => 0106-rfc-process.md} (98%) diff --git a/text/0000-rfc-process.md b/text/0106-rfc-process.md similarity index 98% rename from text/0000-rfc-process.md rename to text/0106-rfc-process.md index 7e30b943e..6f60ead6a 100644 --- a/text/0000-rfc-process.md +++ b/text/0106-rfc-process.md @@ -3,10 +3,10 @@ - Name: RFC Process - Start Date: 2021-05-10 - Author(s): [@ekcasey](https://github.com/ekcasey) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#233](https://github.com/buildpacks/rfcs/pull/233) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: [RFC 0004](0004-rfc-process.md) # Summary From 7beeb1c7a327b19645094e066c0abd75085e9f6d Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Thu, 27 Oct 2022 09:01:52 +0100 Subject: [PATCH 199/372] RFC 0107 [#212] Signed-off-by: Sambhav Kothari --- ...ernatives.md => 0107-pack-buildpack-new-alternatives.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-pack-buildpack-new-alternatives.md => 0107-pack-buildpack-new-alternatives.md} (99%) diff --git a/text/0000-pack-buildpack-new-alternatives.md b/text/0107-pack-buildpack-new-alternatives.md similarity index 99% rename from text/0000-pack-buildpack-new-alternatives.md rename to text/0107-pack-buildpack-new-alternatives.md index b82fbdca1..266e43f17 100644 --- a/text/0000-pack-buildpack-new-alternatives.md +++ b/text/0107-pack-buildpack-new-alternatives.md @@ -3,10 +3,10 @@ - Name: Pack Buildpack New to Support Alternatives - Start Date: 2022-03-10 - Author(s): aidandelaney -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#212](https://github.com/buildpacks/rfcs/pull/212) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: https://github.com/buildpacks/rfcs/blob/main/text/0077-pack-buildpack-create.md # Summary From a713aee1778711fe8cb9ea220824a38ff8712c8a Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Mon, 29 Aug 2022 13:29:38 +0100 Subject: [PATCH 200/372] Allow setting environment variables using the build image Signed-off-by: Sambhav Kothari --- text/0000-build-config.md | 122 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 text/0000-build-config.md diff --git a/text/0000-build-config.md b/text/0000-build-config.md new file mode 100644 index 000000000..737c9353e --- /dev/null +++ b/text/0000-build-config.md @@ -0,0 +1,122 @@ +# Meta +[meta]: #meta +- Name: Build config +- Start Date: 2022-08-29 +- Author(s): [samj1912](https://github.com/samj1912) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +This RFC proposes an easy way to configure build images to allow specifying a `/cnb/config/env.build` CNB environment directory that allows updating the Buildpack `detect` and `build` environment based on the directory. + + +# Definitions +[definitions]: #definitions + +- CNB environment directory: A directory that follows the conventions as defined [here](https://github.com/buildpacks/spec/blob/main/buildpack.md#provided-by-the-buildpacks) +- Operator: Owner of builders and stacks. See [CNB Operator guide](https://buildpacks.io/docs/operator-guide/). + + +# Motivation +[motivation]: #motivation + +Often times, especially in enterprise settings, organizations often have to update the buildpacks to use internal mirrors, proxies and other settings which can be configured easily with environment variables. + +Some examples include - +- `GOPROXY` +- `PIP_INDEX_URL` +- `npm_config_registry` +- `BUNDLE_MIRROR__URL` + +The buildpack logic in the Buildpacks largely remains the same, except these environment variables might need to be injected during the `build` and `detect` phases. + +The environment variables may ideally also take precendence over any user provided values to ensure that the operators have full control over their builders. + +# What it is +[what-it-is]: #what-it-is + +The RFC proposes the introduction of the following directory `/cnb/config/env.build` in build images. The directory follows the same convention as a `CNB environment directory`. The notable difference is that the environment variables sourced from this directory are applied **AFTER** processing the user-provided platform environment variables i.e. they should have the highest precedence. These variables should be available during both `detect` and `build` phases (and the `generate` phase in the future). + + +The operator can define this directory in the build image under `/cnb/config` or `CNB_CONFIG_DIR` if defined. + +# How it Works +[how-it-works]: #how-it-works + +Reference implementation available at https://github.com/buildpacks/lifecycle/pull/899/files + +Examples - + +Buildpack value: `FOO=test` +Build config: `FOO.default=another-value` +Final value: `FOO=test` + + +Buildpack value: `FOO=test` +Build config: `FOO.append=another-value, FOO.delim="` +Final value: `FOO=test:another-value` + +Buildpack value: `FOO=test` +Build config: `FOO.override=another-value` +Final value: `FOO=another-value` + +Buildpack value: `FOO=test` +Platform Enviroment varaible: `FOO=value` +Build config: `FOO=another-value` +Final value: `FOO=value` + +Platform Enviroment varaible: `FOO=value` +Build config: `FOO.override=another-value` +Final value: `FOO=another-value` + +Platform Enviroment varaible: `FOO=value` +Build config: `FOO.prepend=another-value, FOO.delim=:` +Final value: `FOO=another-value:value` + +Platform Enviroment varaible: `FOO=value` +Build config: `FOO.append=another-value, FOO.delim=:` +Final value: `FOO=value:another-value` + +In case a buildpack uses `clear-env=true` then it is up to the buildpack will not see user provided platform values unless it looks in the platform directory and resolve the user provided platform values against the values set by previous buildpacks and the build config. + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + +- Increased complexity in sourcing of environment variables. + +# Alternatives +[alternatives]: #alternatives + + +## Implementing at a Buildpack level + +See https://github.com/paketo-buildpacks/rfcs/pull/241 + +# Prior Art +[prior-art]: #prior-art + +- See https://github.com/paketo-buildpacks/rfcs/pull/241 and +- See the CNB BAT meeting. https://youtu.be/e8FgLwVN5VQ?t=1153 + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +Addition of the definition of the above directory in the Platform specification i.e. - + +- `CNB_CONFIG_DIR` +- `/cnb/config/env.build` From bce8e121ad7795196e13bade078cc0be5884362a Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 4 Nov 2022 10:56:48 -0400 Subject: [PATCH 201/372] Add new field to lifecycle.toml Signed-off-by: Natalie Arellano --- text/0000-launcher-sbom.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-launcher-sbom.md b/text/0000-launcher-sbom.md index 42a15ff3b..d669f67f0 100644 --- a/text/0000-launcher-sbom.md +++ b/text/0000-launcher-sbom.md @@ -104,6 +104,9 @@ The SBOM for a CNB-built image could be broken down as follows: * In theory, there should be no changes needed for end-users to consume the new SBOM files, as the files will be placed in the same directory, with the same structure, as SBOM files for buildpack-provided dependencies. +* To expose the SBOM formats that it supports, a new field `[lifecycle.sbom-formats]` will be added to lifecycle.toml, + where `sbom-formats = [ "" ]` must be supported SBOM media types (just like for buildpack.toml). + # Drawbacks [drawbacks]: #drawbacks From 524c22ce5b9c6a94d20de501a57c0d859240e85f Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Sat, 5 Nov 2022 15:55:29 +0000 Subject: [PATCH 202/372] RFC 0108 [#234] Signed-off-by: Sambhav Kothari --- ...role.md => 0108-governance-component-maintainer-role.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-governance-component-maintainer-role.md => 0108-governance-component-maintainer-role.md} (98%) diff --git a/text/0000-governance-component-maintainer-role.md b/text/0108-governance-component-maintainer-role.md similarity index 98% rename from text/0000-governance-component-maintainer-role.md rename to text/0108-governance-component-maintainer-role.md index f0a1ca388..e5d50fdf7 100644 --- a/text/0000-governance-component-maintainer-role.md +++ b/text/0108-governance-component-maintainer-role.md @@ -3,10 +3,10 @@ - Name: add component maintainer role - Start Date: 2022-09-39 - Author(s): @jjbustamante -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#234](https://github.com/buildpacks/rfcs/pull/234) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/community#196](https://github.com/buildpacks/community/issues/196) - Supersedes: [#228](https://github.com/buildpacks/rfcs/pull/228) # Summary From d25a8e9a2a2f5a5d42a24a31181ffabf1a8b98be Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 9 Nov 2022 14:51:17 -0500 Subject: [PATCH 203/372] adding current kpack maintainers table Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index e840c81ad..b405e8053 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -287,6 +287,14 @@ How do migrate roles and responsibilities into the CNB governance process? Currently the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team) already has a **team lead** assigned and, by definition, each team can have only one **team lead**. In order to provide the current [kpack](https://github.com/pivotal/kpack/) team with the same accountability for the migrated repositories the proposal is to follow the guidelines describe on the [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) +The [kpack's](https://github.com/pivotal/kpack/) maintainers that will be nominated as **component maintainer** in CNB are: + +| Name | Github account | Organization | +| --- | --- | --- | +| Matthew McNew| [@matthewmcnew](https://github.com/matthewmcnew)| VMware| +| Tyler Phelan | [@tylerphelan](https://github.com/tylerphelan)| VMware| +| Tom Kennedy | [@tomkennedy513](https://github.com/tomkennedy513) | VMware | + #### RFC process Once the migration is completed, [kpack](https://github.com/pivotal/kpack/) will follow the [RFC process](https://github.com/buildpacks/rfcs) and [RFC template](https://github.com/buildpacks/rfcs/blob/main/0000-template.md) stablished in CNB project for any new RFC created in the project. @@ -316,7 +324,7 @@ The proposals are: - It's not clear how to handle the budget required to finance the infrastructure to rebuild the CI/CD pipelines on CNCF CNB infrastructure. - Evaluate any legal requirement from [CNCF](https://www.cncf.io) that must be fulfilled before accepting the project into the [CNB](https://buildpacks.io/) ecosystem. -- Relying on the approval of [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) to guarantee current [kpack](https://github.com/pivotal/kpack/) maintainers could keep supporting the project after the donation. + # Drawbacks From eb565bb16251ac4a2cd90211206a2a04fb7fcd25 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 9 Nov 2022 17:34:28 -0500 Subject: [PATCH 204/372] Adding some feedback from Joe Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index b405e8053..fa905b2ac 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -230,6 +230,13 @@ The _spec_ key is used to define the desired state of the [Image](https://github - _builder_: Configuration of the [builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md) resource the image builds will use. - source: The source code that will be monitored/built into images. +# Adopters + +[VMware](https://www.vmware.com/) is the most obvious company using [kpack](https://github.com/pivotal/kpack/), but there are also other companies like: + +- [Bloomberg](http://techatbloomberg.com/opensource) +- [WPengine](https://wpengine.com/) + # Migration ### Repositories @@ -322,10 +329,10 @@ The proposals are: # Risks +- So far the main company behind [kpack](https://github.com/pivotal/kpack/) is [VMware](https://www.vmware.com/), a reduction in the investment from [VMware](https://www.vmware.com/) would create a problem and the CNB project would have to either sunset [kpack](https://github.com/pivotal/kpack/) or find investment from the community. - It's not clear how to handle the budget required to finance the infrastructure to rebuild the CI/CD pipelines on CNCF CNB infrastructure. - Evaluate any legal requirement from [CNCF](https://www.cncf.io) that must be fulfilled before accepting the project into the [CNB](https://buildpacks.io/) ecosystem. - # Drawbacks Why should we _not_ do this? From 87342fd67efd7d84e6c831b804c68e6e5c6202c4 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Sun, 13 Nov 2022 18:24:12 +0000 Subject: [PATCH 205/372] Fix RFC 109 metadata --- text/{0000-build-config.md => 0109-build-config.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-build-config.md => 0109-build-config.md} (92%) diff --git a/text/0000-build-config.md b/text/0109-build-config.md similarity index 92% rename from text/0000-build-config.md rename to text/0109-build-config.md index 737c9353e..d531e447e 100644 --- a/text/0000-build-config.md +++ b/text/0109-build-config.md @@ -3,10 +3,10 @@ - Name: Build config - Start Date: 2022-08-29 - Author(s): [samj1912](https://github.com/samj1912) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#230](https://github.com/buildpacks/rfcs/pull/230) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/lifecycle#956](https://github.com/buildpacks/lifecycle/issues/956), [buildpacks/spec#330](https://github.com/buildpacks/lifecycle/issues/330), [buildpacks/docs#531](https://github.com/buildpacks/lifecycle/issues/531) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 01409d1c6b83132477e9392f1838403942613088 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 13:06:51 -0400 Subject: [PATCH 206/372] First pass Signed-off-by: Natalie Arellano --- text/0000-deprecate-apis.md | 134 ++++++++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+) create mode 100644 text/0000-deprecate-apis.md diff --git a/text/0000-deprecate-apis.md b/text/0000-deprecate-apis.md new file mode 100644 index 000000000..6bd7d7896 --- /dev/null +++ b/text/0000-deprecate-apis.md @@ -0,0 +1,134 @@ +# Meta + +[meta]: #meta + +- Name: Deprecate old buildpack and platform APIs +- Start Date: 2022-06-28 +- Author(s): natalieparellano +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary + +[summary]: #summary + +Today, the CNB project defines XXX buildpack APIs and XXX platform APIs. The CNB lifecycle, for backwards compatibility +and in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all +XXX APIs and all XXX combinations of buildpack and platform APIs. This is a maintenance burden. + +Additionally, as we've talked about adding new features to the project, it has become difficult to determine how these +additions might be compatible with older APIs. This has slowed down progress within the project. + +This RFC proposes: + +* Marking as deprecated the following APIs: + * buildpack APIs XXX and older + * platform APIs XXX and older +* A general policy of: + * Removing support for deprecated APIs 6 months after those APIs are marked as deprecated + +# Definitions + +[definitions]: #definitions + +Buildpack API: XXX + +Platform API: XXX + +lifecycle: XXX + +# Motivation + +[motivation]: #motivation + +- Why should we do this? Lower maintenance burden, enable faster development of new APIs. +- What use cases does it support? As a lifecycle maintainer, I only want to support the latest buildpack and platform + APIs. As a CNB contributor, when thinking about new features, I want a smaller set of supported APIs to care about. +- What is the expected outcome? Hopefully, not too much inconvenience for buildpack authors and platform operators, + because: + - Hopefully they are already on newer versions of the APIs, but if not... + - We will have socialized this change appropriately (e.g., in Slack, through the mailing list, GitHub, etc.) + - The `CNB_DEPRECATION_MODE` setting will have alerted them that upgrading is necessary + - We will have published migration guides to help them upgrade + - 6 months is a reasonable amount of time to complete this process + +# What it is + +[what-it-is]: #what-it-is + +APIs will be marked as deprecated here: + +``` +XXX +``` + +As described in XXX: + +* If `CNB_DEPRECATION_MODE` is set, XXX. +* If `CNB_DEPRECATION_MODE` is unset, XXX. + +After 6 months, the values will be: + +``` +XXX +``` + +* If a buildpack tries to use an unsupported API, the lifecycle will fail with the following message: XXX. +* If a platform tries to use an unsupported API, the lifecycle will fail with the following message: XXX. + +# Migration + +[migration]: #migration + +### Buildpack API + +Buildpack authors will need to update buildpacks using the old APIs to a newer API. These buildpacks will need to be +re-published and re-discovered. + +Platform authors / builder authors will need to re-create builders using the newer buildpacks. + +End-users will need to consumer the newer buildpacks and / or builders. + +### Platform API + +Platform authors will need to update their platform implementations to use the newer platform API. The appropriate value +of `CNB_PLATFORM_API` must be set in the lifecycle's execution environment. + +End-users will need to update their usage of the newer images if they were formerly using a platform API < 0.XXX (see +XXX). + +# Drawbacks + +[drawbacks]: #drawbacks + +Why should we *not* do this? + +* This will inevitably cause extra work for a fraction of buildpacks users. + +# Alternatives + +[alternatives]: #alternatives + +- What other designs have been considered? Doing nothing. +- Why is this proposal the best? If we want to make progress toward a 1.0 version of the spec, we need to start dropping + older APIs. +- What is the impact of not doing this? Slower progress in the project due to the maintenance burden and difficulty + conceptualizing compatibility concerns for so many APIs. + +# Prior Art + +[prior-art]: #prior-art + +golang support schedule: XXX + +RFC that was put "on pause" due to difficulties managing API complexities: XXX + +# Spec. Changes (OPTIONAL) + +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. + +We should clearly mark the deprecated specs as deprecated on their respective branches. From 37ce84da501bd55a676f41d58e844e787efc7dbc Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 17:16:49 -0400 Subject: [PATCH 207/372] Added some more detail Signed-off-by: Natalie Arellano --- text/0000-deprecate-apis.md | 100 +++++++++++++++++++++++++++--------- 1 file changed, 77 insertions(+), 23 deletions(-) diff --git a/text/0000-deprecate-apis.md b/text/0000-deprecate-apis.md index 6bd7d7896..500011369 100644 --- a/text/0000-deprecate-apis.md +++ b/text/0000-deprecate-apis.md @@ -15,18 +15,20 @@ [summary]: #summary -Today, the CNB project defines XXX buildpack APIs and XXX platform APIs. The CNB lifecycle, for backwards compatibility -and in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all -XXX APIs and all XXX combinations of buildpack and platform APIs. This is a maintenance burden. +Today, the CNB project defines 7 buildpack APIs and 7 platform APIs. The CNB lifecycle, for backwards compatibility and +in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all 14 +APIs and all 49 combinations of buildpack and platform APIs. This is a maintenance burden. Additionally, as we've talked about adding new features to the project, it has become difficult to determine how these additions might be compatible with older APIs. This has slowed down progress within the project. +As we progress toward 1.0, it seems prudent to start deprecating APIs from which we have made breaking changes. + This RFC proposes: * Marking as deprecated the following APIs: - * buildpack APIs XXX and older - * platform APIs XXX and older + * buildpack APIs 0.6 and older + * platform APIs 0.6 and older * A general policy of: * Removing support for deprecated APIs 6 months after those APIs are marked as deprecated @@ -34,22 +36,45 @@ This RFC proposes: [definitions]: #definitions -Buildpack API: XXX +**lifecycle**: software that orchestrates a CNB build; it executes in a series of phases that each have a distinct +responsibility + +The CNB project distributes the lifecycle in two ways: + +* As a **lifecycle image**: a minimal [image](https://hub.docker.com/r/buildpacksio/lifecycle) containing the lifecycle + binaries +* As a **lifecycle tarball**: a [`.tgz` file](https://github.com/buildpacks/lifecycle/releases) containing the lifecycle + binaries + +The [**lifecycle +descriptor**](https://github.com/buildpacks/rfcs/blob/main/text/0049-multi-api-lifecycle-descriptor.md#lifecycle-descriptor) +is data describing the lifecycle and the APIs that it supports. It is contained in the `io.buildpacks.lifecycle.apis` +and `io.buildpacks.lifecycle.version` labels on lifecycle images, and the `lifecycle.toml` file in lifecycle tarballs. -Platform API: XXX +**platform**: system or software that orchestrates the lifecycle by invoking each lifecycle phase in order -lifecycle: XXX +**Buildpack API**: specifies the interface between a lifecycle program and one or more buildpacks + +**Platform API**: specifies the interface between a lifecycle program and a platform # Motivation [motivation]: #motivation - Why should we do this? Lower maintenance burden, enable faster development of new APIs. + - We should draw the boundary at the 0.6 APIs because: + - They are both over a year old (March 2021) + - For buildpacks, the 0.6 API is the last API with unstandardized SBOMs. + - For platforms, the 0.6 API is the last API where `detect` happens before `analyze`. + - What use cases does it support? As a lifecycle maintainer, I only want to support the latest buildpack and platform APIs. As a CNB contributor, when thinking about new features, I want a smaller set of supported APIs to care about. + - What is the expected outcome? Hopefully, not too much inconvenience for buildpack authors and platform operators, because: - - Hopefully they are already on newer versions of the APIs, but if not... + - Hopefully they are already on newer versions of the APIs ( + our [user survey](https://docs.google.com/presentation/d/10CBBld2VV0iCfrYPMbFI4--kh9UZ9mJVXhE1p1Cjqls/edit#slide=id.p) + supports this), but if not... - We will have socialized this change appropriately (e.g., in Slack, through the mailing list, GitHub, etc.) - The `CNB_DEPRECATION_MODE` setting will have alerted them that upgrading is necessary - We will have published migration guides to help them upgrade @@ -59,25 +84,50 @@ lifecycle: XXX [what-it-is]: #what-it-is -APIs will be marked as deprecated here: +`io.buildpacks.lifecycle.apis` would contain the following: ``` -XXX +{ + "buildpack": { + "deprecated": [ "0.2", "0.3", "0.4", "0.5", "0.6" ], + "supported": [ "0.2", "0.3", "0.4", "0.5", "0.6", "0.7", "0.8" ] + }, + "platform": { + "deprecated": [ "0.3", "0.4", "0.5", "0.6" ], + "supported": [ "0.3", "0.4", "0.5", "0.6", "0.7", "0.8", "0.9" ] + } +} ``` -As described in XXX: +As described +in [RFC 0049](https://github.com/buildpacks/rfcs/blob/main/text/0049-multi-api-lifecycle-descriptor.md#lifecycle-descriptor) +, if a buildpack or platform tries to use a deprecated API: -* If `CNB_DEPRECATION_MODE` is set, XXX. -* If `CNB_DEPRECATION_MODE` is unset, XXX. +* If `CNB_DEPRECATION_MODE` is unset, the lifecycle will print a warning and continue +* If `CNB_DEPRECATION_MODE`=`warn`, the lifecycle will print a warning and continue +* If `CNB_DEPRECATION_MODE`=`error`, the lifecycle will fail +* If `CNB_DEPRECATION_MODE`=`silent`, the lifecycle will continue w/o warning -After 6 months, the values will be: +After 6 months, the deprecated APIs would become unsupported and `io.buildpacks.lifecycle.apis` would contain the +following: ``` -XXX +{ + "buildpack": { + "deprecated": [ ], + "supported": [ "0.7", "0.8", "" ] + }, + "platform": { + "deprecated": [ ], + "supported": [ "0.7", "0.8", "0.9", "" ] + } +} ``` -* If a buildpack tries to use an unsupported API, the lifecycle will fail with the following message: XXX. -* If a platform tries to use an unsupported API, the lifecycle will fail with the following message: XXX. +* If a buildpack tries to use an unsupported API, the lifecycle will fail with a message such + as: `buildpack API version '0.6' is incompatible with the lifecycle`. +* If a platform tries to use an unsupported API, the lifecycle will fail with a message such + as: `platform API version '0.6' is incompatible with the lifecycle`. # Migration @@ -97,8 +147,9 @@ End-users will need to consumer the newer buildpacks and / or builders. Platform authors will need to update their platform implementations to use the newer platform API. The appropriate value of `CNB_PLATFORM_API` must be set in the lifecycle's execution environment. -End-users will need to update their usage of the newer images if they were formerly using a platform API < 0.XXX (see -XXX). +End-users will need to update their usage of the newer images if they were formerly using a platform +API < [0.4](https://github.com/buildpacks/spec/releases/tag/platform%2Fv0.4) (see +[RFC 0045](https://github.com/buildpacks/rfcs/blob/main/text/0045-launcher-arguments.md)). # Drawbacks @@ -122,9 +173,12 @@ Why should we *not* do this? [prior-art]: #prior-art -golang support schedule: XXX - -RFC that was put "on pause" due to difficulties managing API complexities: XXX +* [Ubuntu release cycle](https://ubuntu.com/about/release-cycle) +* [Golang release policy](https://go.dev/doc/devel/release#policy) +* [Docker Community Edition release cadence](https://www.serverwatch.com/server-news/docker-18-06-ce-debuts-alongside-new-release-cadence/) +* [Kubernetes support window](https://kubernetes.io/blog/2020/08/31/kubernetes-1-19-feature-one-year-support/) +* CNB RFC that was put "on pause" due to difficulties managing API + complexities: https://github.com/buildpacks/rfcs/pull/145 # Spec. Changes (OPTIONAL) From b7f6de674cde8ce60dff7176da58a7ebd2a0f3eb Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 28 Jun 2022 17:23:40 -0400 Subject: [PATCH 208/372] Added some links Signed-off-by: Natalie Arellano --- text/0000-deprecate-apis.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/text/0000-deprecate-apis.md b/text/0000-deprecate-apis.md index 500011369..9aabbe3f3 100644 --- a/text/0000-deprecate-apis.md +++ b/text/0000-deprecate-apis.md @@ -15,9 +15,10 @@ [summary]: #summary -Today, the CNB project defines 7 buildpack APIs and 7 platform APIs. The CNB lifecycle, for backwards compatibility and -in order to enable buildpacks and platforms to upgrade their respective APIs independently, currently supports all 14 -APIs and all 49 combinations of buildpack and platform APIs. This is a maintenance burden. +Today, the CNB project [defines](https://github.com/buildpacks/spec/releases) 7 buildpack APIs and 7 platform APIs. The +CNB lifecycle, for backwards compatibility and in order to enable buildpacks and platforms to upgrade their respective +APIs independently, currently supports all 14 APIs and all 49 combinations of buildpack and platform APIs. This is a +maintenance burden. Additionally, as we've talked about adding new features to the project, it has become difficult to determine how these additions might be compatible with older APIs. This has slowed down progress within the project. @@ -36,7 +37,7 @@ This RFC proposes: [definitions]: #definitions -**lifecycle**: software that orchestrates a CNB build; it executes in a series of phases that each have a distinct +[**lifecycle**](https://github.com/buildpacks/lifecycle): software that orchestrates a CNB build; it executes in a series of phases that each have a distinct responsibility The CNB project distributes the lifecycle in two ways: @@ -53,9 +54,9 @@ and `io.buildpacks.lifecycle.version` labels on lifecycle images, and the `lifec **platform**: system or software that orchestrates the lifecycle by invoking each lifecycle phase in order -**Buildpack API**: specifies the interface between a lifecycle program and one or more buildpacks +[**Buildpack API**](https://github.com/buildpacks/spec/blob/main/buildpack.md): specifies the interface between a lifecycle program and one or more buildpacks -**Platform API**: specifies the interface between a lifecycle program and a platform +[**Platform API**](https://github.com/buildpacks/spec/blob/main/platform.md): specifies the interface between a lifecycle program and a platform # Motivation @@ -77,7 +78,7 @@ and `io.buildpacks.lifecycle.version` labels on lifecycle images, and the `lifec supports this), but if not... - We will have socialized this change appropriately (e.g., in Slack, through the mailing list, GitHub, etc.) - The `CNB_DEPRECATION_MODE` setting will have alerted them that upgrading is necessary - - We will have published migration guides to help them upgrade + - We will have published [migration guides](https://buildpacks.io/docs/reference/spec/migration/) to help them upgrade - 6 months is a reasonable amount of time to complete this process # What it is From 48644de17d1f7a634956f8584c3a0341053ac629 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 15 Nov 2022 13:19:15 -0500 Subject: [PATCH 209/372] RFC 0110 [#226] Signed-off-by: Natalie Arellano --- text/{0000-deprecate-apis.md => 0110-deprecate-apis.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-deprecate-apis.md => 0110-deprecate-apis.md} (98%) diff --git a/text/0000-deprecate-apis.md b/text/0110-deprecate-apis.md similarity index 98% rename from text/0000-deprecate-apis.md rename to text/0110-deprecate-apis.md index 9aabbe3f3..a44ac3bfc 100644 --- a/text/0000-deprecate-apis.md +++ b/text/0110-deprecate-apis.md @@ -5,10 +5,10 @@ - Name: Deprecate old buildpack and platform APIs - Start Date: 2022-06-28 - Author(s): natalieparellano -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#226](https://github.com/buildpacks/rfcs/pull/226) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: N/A # Summary From d6820b8ecbddf22167f5fe662624fd777349407a Mon Sep 17 00:00:00 2001 From: wanjunlei Date: Thu, 18 Aug 2022 14:15:50 +0800 Subject: [PATCH 210/372] RFC to support push image to isecure registries Signed-off-by: wanjunlei --- text/0000-support-insecure-registries.md | 72 ++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 text/0000-support-insecure-registries.md diff --git a/text/0000-support-insecure-registries.md b/text/0000-support-insecure-registries.md new file mode 100644 index 000000000..50d1d7aeb --- /dev/null +++ b/text/0000-support-insecure-registries.md @@ -0,0 +1,72 @@ +# Meta +[meta]: #meta +- Name: Support for pull images from or push images to insecure image registries. +- Start Date: 2022-08-17 +- Author(s): wanjunlei +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This RFC describes how to pull images form or push images to insecure image registries when build images using buildpacks. + +# Definitions +[definitions]: #definitions + +Insecure image registry - The insecure image registry mentioned in this RFC refers to the image registry using the http protocol, and with a non-internal ip or a domain name. Because currently buildpacks can access inecure image registries using internal ip. + +# Motivation +[motivation]: #motivation + +To fix issue [Support insecure registries in non-daemon case](https://github.com/buildpacks/lifecycle/issues/524). + +# What it is +[what-it-is]: #what-it-is + +With this RFC, user can use images in insecure image registries to build image, and push the compiled image to the insecure image registries like this. + +```shell +creator --run-image=develoment-registry.com/run-java:v16 --insecure-registry=develoment-registry.com --insecure-registry=testing-registry.com testing-registry.com/java-sample:latest +``` + +The flag `--insecure-registry` will add to analyzer, export, restor, rebaser and image tool too. + +# How it Works +[how-it-works]: #how-it-works + +With this [PR](https://github.com/buildpacks/imgutil/pull/154), the component that buildpacks used to operate images, already supports pulling images from or pushing images to insecure registries. +We should create a image with insecure registry by calling [NewImage](https://github.com/buildpacks/imgutil/blob/main/remote/remote.go#L119) like this + +```shell +remote.NewImage(imageName, authn.DefaultKeychain, withRegistrySetting("mydomain.registry.com:1080", true, false) +``` +> The second parameter specify whether the registry is insecure. +> The third parameter can be always false. + +It is necessary to judge whether the registry where this image is located is insecure. If so, it needs to set the insecure registry through `WithRegistrySetting`. + +The `NewImage` can set base image and prev image using `FromBaseImage` and `WithPreviousImage`, so it may need to call `WithRegistrySetting` multiple times to set up multiple insecure registries. + +# Drawbacks +[drawbacks]: #drawbacks + +N/A + +# Alternatives +[alternatives]: #alternatives + +N/A + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +A new flag `--insecure-registry` will add to analyzer, creator, export, restor, rebaser and image tool. \ No newline at end of file From c7fc5674b3758d21bdb330e3c079298911f19026 Mon Sep 17 00:00:00 2001 From: wanjunlei Date: Fri, 26 Aug 2022 09:07:49 +0800 Subject: [PATCH 211/372] resolve conversation Signed-off-by: wanjunlei --- text/0000-support-insecure-registries.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/text/0000-support-insecure-registries.md b/text/0000-support-insecure-registries.md index 50d1d7aeb..dac574317 100644 --- a/text/0000-support-insecure-registries.md +++ b/text/0000-support-insecure-registries.md @@ -12,18 +12,20 @@ # Summary [summary]: #summary -This RFC describes how to pull images form or push images to insecure image registries when build images using buildpacks. +This RFC describes how to pull images from or push images to insecure image registries when building images using buildpacks. # Definitions [definitions]: #definitions -Insecure image registry - The insecure image registry mentioned in this RFC refers to the image registry using the http protocol, and with a non-internal ip or a domain name. Because currently buildpacks can access inecure image registries using internal ip. +Insecure image registry - The insecure image registry mentioned in this RFC refers to the image registry using the http protocol, and with a non-internal ip or a domain name. Because currently buildpacks can access insecure image registries using internal ip. # Motivation [motivation]: #motivation To fix issue [Support insecure registries in non-daemon case](https://github.com/buildpacks/lifecycle/issues/524). +To pull images from insecure image registries when building images using buildpacks, and push target images to insecure image registries after the build is complete. + # What it is [what-it-is]: #what-it-is From 95a97d75cea4e16fdb56004ab2cdf333784abde3 Mon Sep 17 00:00:00 2001 From: wanjunlei Date: Tue, 30 Aug 2022 09:25:28 +0800 Subject: [PATCH 212/372] resolve conversation Signed-off-by: wanjunlei --- text/0000-support-insecure-registries.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-support-insecure-registries.md b/text/0000-support-insecure-registries.md index dac574317..d691fe5dd 100644 --- a/text/0000-support-insecure-registries.md +++ b/text/0000-support-insecure-registries.md @@ -35,7 +35,7 @@ With this RFC, user can use images in insecure image registries to build image, creator --run-image=develoment-registry.com/run-java:v16 --insecure-registry=develoment-registry.com --insecure-registry=testing-registry.com testing-registry.com/java-sample:latest ``` -The flag `--insecure-registry` will add to analyzer, export, restor, rebaser and image tool too. +The flag `--insecure-registry` will be added to analyzer, export, restorer, rebaser and image tool too. # How it Works [how-it-works]: #how-it-works @@ -71,4 +71,4 @@ N/A # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes -A new flag `--insecure-registry` will add to analyzer, creator, export, restor, rebaser and image tool. \ No newline at end of file +A new flag `--insecure-registry` will add to analyzer, creator, export, restorer, rebaser and image tool. \ No newline at end of file From 3774552232b9a9a5d1a548e99147c7fd5e5baf84 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 15 Nov 2022 13:20:31 -0500 Subject: [PATCH 213/372] RFC 0111 [#229] Signed-off-by: Natalie Arellano --- ...re-registries.md => 0111-support-insecure-registries.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-support-insecure-registries.md => 0111-support-insecure-registries.md} (95%) diff --git a/text/0000-support-insecure-registries.md b/text/0111-support-insecure-registries.md similarity index 95% rename from text/0000-support-insecure-registries.md rename to text/0111-support-insecure-registries.md index d691fe5dd..cbc7f3ba4 100644 --- a/text/0000-support-insecure-registries.md +++ b/text/0111-support-insecure-registries.md @@ -3,10 +3,10 @@ - Name: Support for pull images from or push images to insecure image registries. - Start Date: 2022-08-17 - Author(s): wanjunlei -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#229](https://github.com/buildpacks/rfcs/pull/229) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: N/A # Summary From 1fe522863c254ac4d9863c29cb9a96c311af9aba Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 15 Nov 2022 13:23:14 -0500 Subject: [PATCH 214/372] RFC 0112 [#225] Signed-off-by: Natalie Arellano --- text/{0000-launcher-sbom.md => 0112-launcher-sbom.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-launcher-sbom.md => 0112-launcher-sbom.md} (98%) diff --git a/text/0000-launcher-sbom.md b/text/0112-launcher-sbom.md similarity index 98% rename from text/0000-launcher-sbom.md rename to text/0112-launcher-sbom.md index d669f67f0..bb2080381 100644 --- a/text/0000-launcher-sbom.md +++ b/text/0112-launcher-sbom.md @@ -5,10 +5,10 @@ - Name: SBOM for lifecycle / launcher - Start Date: 2022-06-27 - Author(s): natalieparellano -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#225](https://github.com/buildpacks/rfcs/pull/225) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: N/A # Summary From 18b926cb784a57c13c5b157023a745e1b173f768 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 30 Nov 2022 14:06:43 -0500 Subject: [PATCH 215/372] adding feedback from Joe Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index fa905b2ac..1f8d82462 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -230,12 +230,27 @@ The _spec_ key is used to define the desired state of the [Image](https://github - _builder_: Configuration of the [builder](https://github.com/pivotal/kpack/blob/main/docs/builders.md) resource the image builds will use. - source: The source code that will be monitored/built into images. +# Contributors + +Contributions to [kpack](https://github.com/pivotal/kpack/) during the period 2022-2019 can be summarized as follow + +```mermaid +pie showData + title Pull Requests Open or Closed + "VMWare or Pivotal" : 438 + "Others" : 37 +``` + # Adopters -[VMware](https://www.vmware.com/) is the most obvious company using [kpack](https://github.com/pivotal/kpack/), but there are also other companies like: +[VMware](https://www.vmware.com/) is the most obvious company using [kpack](https://github.com/pivotal/kpack/) as a core component in their [Tanzu](https://tanzu.vmware.com/tanzu) offer. [kpack](https://github.com/pivotal/kpack/) has been used in the enterprise for years in hundreds of costumers. + +Some others companies using [kpack](https://github.com/pivotal/kpack/) are: - [Bloomberg](http://techatbloomberg.com/opensource) - [WPengine](https://wpengine.com/) +- [Terasky](https://www.terasky.com/) +- [SAP](https://www.sap.com/) # Migration @@ -299,8 +314,19 @@ The [kpack's](https://github.com/pivotal/kpack/) maintainers that will be nomina | Name | Github account | Organization | | --- | --- | --- | | Matthew McNew| [@matthewmcnew](https://github.com/matthewmcnew)| VMware| -| Tyler Phelan | [@tylerphelan](https://github.com/tylerphelan)| VMware| | Tom Kennedy | [@tomkennedy513](https://github.com/tomkennedy513) | VMware | +| Yael Harel | [@yaelharel](https://github.com/yaelharel) | VMware| +| Daniel Chen |[@chenbh](https://github.com/chenbh) | VMware| +| Juan Bustamante |[@jjbustamante](https://github.com/jjbustamante) | VMware| + +Also those members are willing to become more involved with CNB projects and become **Platform maintainers** in the near future. + +Outside VMware, the following contributors manifested their desired to become [kpack's](https://github.com/pivotal/kpack/) **component maintainer**. + +| Name | Github account | Organization | +| --- | --- | --- | +| Sambhav Kothari| [@samj1912](https://github.com/samj1912) | Bloomberg | +| Aidan Delaney| [@AidanDelaney](https://github.com/AidanDelaney) | Bloomberg | #### RFC process From 2a340cbf6642f987e6ea5d95014f56edd841bc8d Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 1 Dec 2022 11:36:14 -0600 Subject: [PATCH 216/372] RFC: Merge Distribution Team with Platform Team Signed-off-by: Joe Kutner --- text/0000-distribution-team-merge-platform.md | 104 ++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 text/0000-distribution-team-merge-platform.md diff --git a/text/0000-distribution-team-merge-platform.md b/text/0000-distribution-team-merge-platform.md new file mode 100644 index 000000000..53a4917a6 --- /dev/null +++ b/text/0000-distribution-team-merge-platform.md @@ -0,0 +1,104 @@ +# Meta +[meta]: #meta +- Name: Merge Distribution Team with Platform Team +- Start Date: 2022-12-01 +- Author(s): [@jkutner](https://github.com/jkutner) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: [RFC-0062](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) + +# Summary +[summary]: #summary + +This is a proposal to consolidate two existing teams, Distribution and Platform, into a single team. + +# Definitions +[definitions]: #definitions + +* Platform - pack and any other platforms or platform components maintained by the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team) +* Distribution - tools and services that support the distribution, discovery, and integration of buildpacks. Owned by the [Distribution Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#distribution-team) + +# Motivation +[motivation]: #motivation + +The distribution team only has one maintainer, which is not sustainable and present a redunancy risk. However, the distribution team's workload is fairly small, consisting mostly of security patches and simple releases. At the same time, the Platform team has lost key resources including a maintainer and team lead. The components owned by the Distribution team align well with the Platform team's mission. For these reasons, we aim to consolidate leadership energy by merging these teams. + +# What it is +[what-it-is]: #what-it-is + +The active maintainers and contributors of the Distribution Team will move into the Platform team with the same role. The components owned by the Distribution Team will henceforth be owned by the Platform Team. A single Team Lead will continue to represent Platform Team. + +The Distribution Team will be removed from the CNB [Governance model](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md). + +# How it Works +[how-it-works]: #how-it-works + +The Platform Team will have the following members: + +* Team Lead: Terence Lee +* Maintainers: Terence Lee, Joe Kutner, David Freilich +* Contributors: All active contributors from both existing teams + +The Platform Team will own the following components: +* [pack](https://github.com/buildpacks/pack) +* [Tekton Tasks + Pipelines](https://github.com/buildpacks/tekton-integration) +* [CircleCI Pack Orb](https://github.com/buildpacks/pack-orb) +* [Buildpack Registry API](https://github.com/buildpacks/registry-api) +* [Buildpack Registry Index](https://github.com/buildpacks/registry-index) +* [Buildpack Registry Namespace Owners](https://github.com/buildpacks/registry-namespaces) +* [Github Actions](https://github.com/buildpacks/github-actions) + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + +* This increases the surface of the Platform Team, which already suffered from attrition. It is also one the team that owns one of the most critical CNB components: pack. This could put additional strain on its members. + +# Alternatives +[alternatives]: #alternatives + +* Do nothing - keep the same team structure. Both the Platform team and Distribution team would lack redundancy +* Disolve the Distribution Team - sunset the components it owns and find alternatives to Buildpack Registry and github-actions + +# Prior Art +[prior-art]: #prior-art + +* [Create a Distribution Team](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) +* [Announcing OpenTelemetry: the merger of OpenCensus and OpenTracing](https://cloudblogs.microsoft.com/opensource/2019/05/23/announcing-opentelemetry-cncf-merged-opencensus-opentracing/) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +TBD + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +N/A + +# History +[history]: #history + + From d65d34f81fd7f7b47af7fb052293403541ea2b81 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 1 Dec 2022 14:19:20 -0500 Subject: [PATCH 217/372] Add `Implemented` as a valid RFC status. Signed-off-by: Natalie Arellano --- text/0094-add-status-field.md | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/text/0094-add-status-field.md b/text/0094-add-status-field.md index 5dd946abb..0a477ad23 100644 --- a/text/0094-add-status-field.md +++ b/text/0094-add-status-field.md @@ -70,6 +70,12 @@ Since the approval of this RFC additional concerns X, Y, and Z have been raised. If at a later date the core team or responsible sub-team wishes to move the RFC from "On Hold" back to "Approved", this should be done via PR and another note should be added, describing the resolution of the discussion and the rationale for proceeding with the original RFC. +When an RFC is fully implemented (i.e., the associated tracking issue is closed), then change is reflected like so: + +``` +- Status: Implemented +``` + # Drawbacks [drawbacks]: #drawbacks @@ -90,10 +96,33 @@ If at a later date the core team or responsible sub-team wishes to move the RFC [unresolved-questions]: #unresolved-questions - Should **"Implemented"** be one of the accepted values? - - Perhaps, but then what's the process of keeping the _Implemented_ status in sync? + - Yes, the RFC shepherd should create a PR updating the status from `Approved` to `Implemented` when the tracking issue is closed. - How do we retroactively add a **Status:** field to prior RFCs? - Manually? - How do we validate the [#meta](#meta) section of prior RFCs? - Perhaps we make a [../validate-rfcs.sh](../validate-rfcs.sh) script that lints the **#meta** section of all rfcs? + +# History +[history]: #history + +## Amended +### Meta +[meta-1]: #meta-1 +- Name: Add Implemented +- Start Date: 2022-12-01 +- Author(s): natalieparellano +- Amendment Pull Request: (leave blank) + +### Summary + +Added `Implemented` as a valid status. + +### Motivation + +Why was this amendment necessary? + +It can be confusing when reviewing older RFCs to know if they were implemented or not. +This can lead to RFCs being "forgotten". +The introduction of [tracking issues](https://github.com/buildpacks/rfcs/blob/main/.github/ISSUE_TEMPLATE/tracking.md) to our process can help ensure that the status gets updated appropriately. From 75a7e1d62474d2bbf663375a3010bd378bd8e7b4 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 2 Dec 2022 09:07:59 -0600 Subject: [PATCH 218/372] Revert "Additional OCI Artifacts" --- text/0000-additonal-oci-artifacts.md | 90 ---------------------------- 1 file changed, 90 deletions(-) delete mode 100644 text/0000-additonal-oci-artifacts.md diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md deleted file mode 100644 index a4f3b7593..000000000 --- a/text/0000-additonal-oci-artifacts.md +++ /dev/null @@ -1,90 +0,0 @@ -# Meta -[meta]: #meta -- Name: Additional OCI Artifacts from Buildpacks -- Start Date: 2022-04-24 -- Author(s): agracey -- Status: Draft -- RFC Pull Request: (leave blank) -- CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) - -# Summary -[summary]: #summary - -Allow buildpack authors to specify additional artifacts from their buildpacks to be uploaded to a registry along with the standard set of layers. - -# Definitions -[definitions]: #definitions - - -- WebAssembly (WASM) -- Portable binaries to be run with a WebAssembly virtual machine. -- [OCI Artifact](https://github.com/opencontainers/artifacts/blob/main/definitions-terms.md#media-type) -- Any blob of data that can be stored in an OCI registry. From the spec: -> An artifact has a type, which has a collection of layers. The Artifact is defined as unique by its `manifest.config.mediaType`. Layers are defined by their `layer.config.mediaType`. - -# Motivation -[motivation]: #motivation - -Cloud Native Buildpacks are awesome for building container images but come with a draw back that they can only produce a single output. This is limiting to platform builders because there are many cases where you might want to have multiple outputs from the same build process. - -# What it is -[what-it-is]: #what-it-is - -This change allows for additional flexibility in what buildpack authors and platforms can build on top of buildpacks. Three initial use cases are WebAssembly, test output, and Helm charts. - -For the first, a buildpack would be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or runwasi). - -Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, ConfigMaps to pass around out-of-band, etc...) - -Lastly, it may be advantageous to allow a buildpack to produce the results of unit tests and publish alongside the container itself. Then a tool (such as Kubewarden) could gate running of the container based on the passing tests. - -# How it Works -[how-it-works]: #how-it-works - -Add an optional file per layer called `artifacts.toml` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. - -For the WASM use-case, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following `artifacts.toml`: - -``` -[[artifact]] -tag = "wasm" -file = "mywasmprogram.wasm" # Should be a file relative to the current $LAYERS_DIR - -[[artifact.labels]] -ConfigMediaType = "application/vnd.wasm.config.v1+json" -ContentLayerMediaType = "application/vnd.wasm.content.layer.v1+wasm" -``` - -As a note: to keep this generic to any potential OCI artifact, it seems best to provide a labelling mechanism instead of prescribing certain fields. - -# Migration -[migration]: #migration - -This functionality should be purely additive and not break existing buildpacks. - -# Drawbacks -[drawbacks]: #drawbacks - -This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. This can be mitigated with good docs and platform output. Adding a change to make the standard image output optional seems like a larger change than is appropriate at the moment. - -# Alternatives -[alternatives]: #alternatives - -Other ways of doing this would be to add the artifacts to the primary OCI image directly then pull back out in a future stage. This has two obvious issues: artifact size and escalation in tooling needed in pipeline. - -# Prior Art -[prior-art]: #prior-art - -None that I'm aware of. - -# Unresolved Questions -[unresolved-questions]: #unresolved-questions - -- [] What to do when there are collisions in tags. -- [] Could the [Bindle](https://github.com/deislabs/bindle) project help? - -# Spec. Changes (OPTIONAL) -[spec-changes]: #spec-changes - -The only addition to the spec is the addition of the artifacts.toml as well as the changes to lifecycle to parse and act on the new config. - From 2e94dbf8c726c9fa72add302dd4b265d470d9ef0 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 2 Dec 2022 09:10:52 -0600 Subject: [PATCH 219/372] Revert "Revert "Additional OCI Artifacts"" --- text/0000-additonal-oci-artifacts.md | 90 ++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 text/0000-additonal-oci-artifacts.md diff --git a/text/0000-additonal-oci-artifacts.md b/text/0000-additonal-oci-artifacts.md new file mode 100644 index 000000000..a4f3b7593 --- /dev/null +++ b/text/0000-additonal-oci-artifacts.md @@ -0,0 +1,90 @@ +# Meta +[meta]: #meta +- Name: Additional OCI Artifacts from Buildpacks +- Start Date: 2022-04-24 +- Author(s): agracey +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +Allow buildpack authors to specify additional artifacts from their buildpacks to be uploaded to a registry along with the standard set of layers. + +# Definitions +[definitions]: #definitions + + +- WebAssembly (WASM) -- Portable binaries to be run with a WebAssembly virtual machine. +- [OCI Artifact](https://github.com/opencontainers/artifacts/blob/main/definitions-terms.md#media-type) -- Any blob of data that can be stored in an OCI registry. From the spec: +> An artifact has a type, which has a collection of layers. The Artifact is defined as unique by its `manifest.config.mediaType`. Layers are defined by their `layer.config.mediaType`. + +# Motivation +[motivation]: #motivation + +Cloud Native Buildpacks are awesome for building container images but come with a draw back that they can only produce a single output. This is limiting to platform builders because there are many cases where you might want to have multiple outputs from the same build process. + +# What it is +[what-it-is]: #what-it-is + +This change allows for additional flexibility in what buildpack authors and platforms can build on top of buildpacks. Three initial use cases are WebAssembly, test output, and Helm charts. + +For the first, a buildpack would be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or runwasi). + +Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, ConfigMaps to pass around out-of-band, etc...) + +Lastly, it may be advantageous to allow a buildpack to produce the results of unit tests and publish alongside the container itself. Then a tool (such as Kubewarden) could gate running of the container based on the passing tests. + +# How it Works +[how-it-works]: #how-it-works + +Add an optional file per layer called `artifacts.toml` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. + +For the WASM use-case, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following `artifacts.toml`: + +``` +[[artifact]] +tag = "wasm" +file = "mywasmprogram.wasm" # Should be a file relative to the current $LAYERS_DIR + +[[artifact.labels]] +ConfigMediaType = "application/vnd.wasm.config.v1+json" +ContentLayerMediaType = "application/vnd.wasm.content.layer.v1+wasm" +``` + +As a note: to keep this generic to any potential OCI artifact, it seems best to provide a labelling mechanism instead of prescribing certain fields. + +# Migration +[migration]: #migration + +This functionality should be purely additive and not break existing buildpacks. + +# Drawbacks +[drawbacks]: #drawbacks + +This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. This can be mitigated with good docs and platform output. Adding a change to make the standard image output optional seems like a larger change than is appropriate at the moment. + +# Alternatives +[alternatives]: #alternatives + +Other ways of doing this would be to add the artifacts to the primary OCI image directly then pull back out in a future stage. This has two obvious issues: artifact size and escalation in tooling needed in pipeline. + +# Prior Art +[prior-art]: #prior-art + +None that I'm aware of. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- [] What to do when there are collisions in tags. +- [] Could the [Bindle](https://github.com/deislabs/bindle) project help? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +The only addition to the spec is the addition of the artifacts.toml as well as the changes to lifecycle to parse and act on the new config. + From 618996e901adec3010fe6dfcd37977e6a554cd32 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 2 Dec 2022 10:12:50 -0500 Subject: [PATCH 220/372] Update status of approved RFCs to show if they were implemented Signed-off-by: Natalie Arellano --- text/0001-pack-suggest-stacks.md | 2 +- text/0002-pack-logging-refactor.md | 2 +- text/0003-pack-inspect-image.md | 2 +- text/0005-contractual-build-plan.md | 2 +- text/0006-stage-specific-mixins.md | 2 +- text/0007-spec-distribution.md | 2 +- text/0008-buildpack-config-for-dist.md | 2 +- text/0009-auto-load-user-provided-environment-variables.md | 2 +- text/0010-api-versions.md | 2 +- text/0012-service-binding.md | 2 +- text/0013-app-layer-metadata-source.md | 2 +- text/0014-combined-restorer-analyzer-phases.md | 2 +- text/0015-windows-lifecycle.md | 2 +- text/0016-use-email-more.md | 2 +- text/0017-pack-build-default-process-flag.md | 2 +- text/0018-remove-pack-run.md | 2 +- text/0019-project-descriptor.md | 2 +- text/0020-landing-page.md | 2 +- text/0021-lifecycle-compat-verification.md | 2 +- text/0023-circleci-orb.md | 2 +- text/0024-lifecycle-multicall-binary-for-build.md | 2 +- text/0025-dont-trust-builders.md | 2 +- text/0026-lifecycle-all.md | 2 +- text/0027-spec-api-branches.md | 2 +- text/0028-remove-cred-helpers.md | 2 +- text/0029-template-changes.md | 2 +- text/0030-links-for-buildpacks.md | 2 +- text/0031-bionic-mixins.md | 2 +- text/0032-update-json-cnb-registry.md | 2 +- text/0033-add-author.md | 2 +- text/0034-image-labels.md | 2 +- text/0035-buildpack-versions.md | 2 +- text/0036-cnb-buildpack-directory-env-var.md | 2 +- text/0037-buildpack-uris.md | 2 +- text/0038-gh-repo-labels.md | 2 +- text/0039-release-process.md | 2 +- text/0041-api-version-compat.md | 2 +- text/0042-process-specific-env.md | 2 +- text/0043-increase-build-plan-flexibility.md | 2 +- text/0044-pack-publish-buildpack.md | 2 +- text/0046-pack-pull-policy.md | 2 +- text/0047-danger-zone.md | 2 +- text/0048-inline-buildpack.md | 2 +- text/0050-stack-metadata.md | 2 +- text/0051-override-env-by-default.md | 2 +- text/0052-opt-in-layer-caching.md | 2 +- text/0054-project-descriptor-schema.md | 2 +- text/0055-deprecate-service-bindings.md | 2 +- text/0056-any-stack-buildpacks.md | 2 +- text/0057-exec.d-shell-free-profile-d.md | 2 +- text/0058-pack-subcommands.md | 2 +- text/0059-label-rfcs.md | 2 +- text/0060-create-repo-issues.md | 2 +- text/0062-distribution-team.md | 2 +- text/0063-stack-images-config-env-path-must-exist.md | 2 +- text/0064-buildpacks-contribute-default-process-type.md | 2 +- text/0067-report-toml-image-manifest-size.md | 2 +- text/0068-buildpack-registry-search-api.md | 2 +- text/0070-new-buildpack-toml-keys.md | 2 +- text/0071-cnb-user-research-2021.md | 2 +- text/0072-image-workdir.md | 2 +- text/0074-layer-table.md | 2 +- text/0075-move-analyze-phase.md | 2 +- text/0077-pack-buildpack-create.md | 2 +- text/0078-group-additions.md | 2 +- text/0080-builder-key-in-project-descriptor.md | 2 +- text/0081-process-specific-working-directory.md | 2 +- text/0082-pack-package-refactor.md | 2 +- text/0084-project-descriptor-domains.md | 2 +- text/0085-run-uid.md | 2 +- text/0088-minimum-docs-lifecycle.md | 2 +- text/0089-buildpack-authors-tooling-subteam.md | 2 +- text/0091-pack-cache-options.md | 2 +- text/0092-add-visual-pack-build.md | 2 +- text/0093-remove-shell-processes.md | 2 +- text/0094-add-status-field.md | 2 +- text/0095-sbom.md | 2 +- text/0097-official-utility-buildpacks.md | 1 + text/0098-rfc-issue-generation.md | 2 +- text/0099-slack.md | 2 +- text/0100-buildpack-input-vars.md | 2 +- text/0101-system-buildpacks-in-builder.md | 1 + text/0103-source-date-epoch.md | 2 +- text/0104-dot-profile-utility-buildpack.md | 2 +- text/0105-dockerfiles.md | 1 + text/0106-rfc-process.md | 2 +- 86 files changed, 86 insertions(+), 83 deletions(-) diff --git a/text/0001-pack-suggest-stacks.md b/text/0001-pack-suggest-stacks.md index d4942c8ba..89f508fca 100644 --- a/text/0001-pack-suggest-stacks.md +++ b/text/0001-pack-suggest-stacks.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Suggest Stacks - Start Date: 2019-04-25 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#4](https://github.com/buildpacks/rfcs/pull/4), [pack#190](https://github.com/buildpacks/pack/pull/190) - CNB Issue: [pack#156](https://github.com/buildpacks/pack/issues/156) - Supersedes: N/A diff --git a/text/0002-pack-logging-refactor.md b/text/0002-pack-logging-refactor.md index 10d1fbc2c..73c2eac7f 100644 --- a/text/0002-pack-logging-refactor.md +++ b/text/0002-pack-logging-refactor.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Pack Logging Interface - Start Date: 2019-05-06 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#6](https://github.com/buildpacks/rfcs/pull/6), [pack#182](https://github.com/buildpacks/pack/pull/182) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0003-pack-inspect-image.md b/text/0003-pack-inspect-image.md index 80430a77f..0d2bc65a5 100644 --- a/text/0003-pack-inspect-image.md +++ b/text/0003-pack-inspect-image.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Inspect Image - Start Date: 2019-04-25 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#5](https://github.com/buildpacks/rfcs/pull/5) - CNB Issue: - Supersedes: N/A diff --git a/text/0005-contractual-build-plan.md b/text/0005-contractual-build-plan.md index a6679bee6..c6d1b7a44 100644 --- a/text/0005-contractual-build-plan.md +++ b/text/0005-contractual-build-plan.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Contractual Build Plan - Start Date: 2019-04-12 -- Status: Approved +- Status: Implemented - CNB Pull Requests: [rfcs#12](https://github.com/buildpacks/rfcs/pull/12), [spec#52](https://github.com/buildpacks/spec/pull/52), [lifecycle#149](https://github.com/buildpacks/lifecycle/pull/149) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0006-stage-specific-mixins.md b/text/0006-stage-specific-mixins.md index e58ccecd4..ffa762bda 100644 --- a/text/0006-stage-specific-mixins.md +++ b/text/0006-stage-specific-mixins.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Stage-specific Mixins - Start Date: 2019-06-13 -- Status: Approved +- Status: Superseded - CNB Pull Requests: [rfcs#13](https://github.com/buildpacks/rfcs/pull/13), [spec#54](https://github.com/buildpacks/spec/pull/54) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0007-spec-distribution.md b/text/0007-spec-distribution.md index 2d6118e91..841c747c2 100644 --- a/text/0007-spec-distribution.md +++ b/text/0007-spec-distribution.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Buildpack Distribution Specification - Start Date: 2019-04-12 -- Status: Approved +- Status: Implemented - CNB Pull Requests: [rfcs#11](https://github.com/buildpacks/rfcs/pull/11), [spec#53](https://github.com/buildpacks/spec/pull/53), [lifecycle#149](https://github.com/buildpacks/lifecycle/pull/149), [pack#243](https://github.com/buildpacks/pack/issues/243) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0008-buildpack-config-for-dist.md b/text/0008-buildpack-config-for-dist.md index 7612ecc83..85c01b08c 100644 --- a/text/0008-buildpack-config-for-dist.md +++ b/text/0008-buildpack-config-for-dist.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Buildpack Configuration for Distribution - Start Date: July 22, 2019 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#15](https://github.com/buildpacks/rfcs/pull/15) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0009-auto-load-user-provided-environment-variables.md b/text/0009-auto-load-user-provided-environment-variables.md index 5b9d86ff6..2425464f8 100644 --- a/text/0009-auto-load-user-provided-environment-variables.md +++ b/text/0009-auto-load-user-provided-environment-variables.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Auto-load User-provided Environment Variables - Start Date: 2019-06-17 -- Status: Approved +- Status: Implemented - CNB Pull Requests: [rfcs#14](https://github.com/buildpacks/rfcs/pull/14), [spec#55](https://github.com/buildpacks/spec/pull/55), [lifecycle#163](https://github.com/buildpacks/lifecycle/pull/163) - CNB Issues: (lifecycle issues to follow) diff --git a/text/0010-api-versions.md b/text/0010-api-versions.md index ad0c0d537..b7506a2f6 100644 --- a/text/0010-api-versions.md +++ b/text/0010-api-versions.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: API Versions - Start Date: 2019-08-05 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#19](https://github.com/buildpacks/rfcs/pull/19), [pack#282](https://github.com/buildpacks/pack/pull/282) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0012-service-binding.md b/text/0012-service-binding.md index ce3c22d3e..2716f5f17 100644 --- a/text/0012-service-binding.md +++ b/text/0012-service-binding.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Service Binding - Start Date: 2019-08-06 -- Status: Approved +- Status: Superseded - CNB Pull Request: [rfcs#22](https://github.com/buildpacks/rfcs/pull/22), [spec#57](https://github.com/buildpacks/spec/pull/57) - CNB Issue: - Supersedes: N/A diff --git a/text/0013-app-layer-metadata-source.md b/text/0013-app-layer-metadata-source.md index cd8d7ee85..a999d4b0e 100644 --- a/text/0013-app-layer-metadata-source.md +++ b/text/0013-app-layer-metadata-source.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Manifest - App metadata - add source type, version and metadata - Start Date: 2019-06-10 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#9](https://github.com/buildpacks/rfcs/pull/9) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0014-combined-restorer-analyzer-phases.md b/text/0014-combined-restorer-analyzer-phases.md index 1c48f5384..06f9c2d2e 100644 --- a/text/0014-combined-restorer-analyzer-phases.md +++ b/text/0014-combined-restorer-analyzer-phases.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Lifecycle cache contract changes. - Start Date: 2019-08-02 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0015-windows-lifecycle.md b/text/0015-windows-lifecycle.md index 0177f44f2..0ddf93c42 100644 --- a/text/0015-windows-lifecycle.md +++ b/text/0015-windows-lifecycle.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Support for Windows in the Lifecycle components - Start Date: 2019-10-07 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#27](https://github.com/buildpacks/rfcs/pull/27) - CNB Issue: - Supersedes: N/A diff --git a/text/0016-use-email-more.md b/text/0016-use-email-more.md index eb3940403..a2b445d00 100644 --- a/text/0016-use-email-more.md +++ b/text/0016-use-email-more.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Use Email More - Start Date: 2019-12-05 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0017-pack-build-default-process-flag.md b/text/0017-pack-build-default-process-flag.md index cb98f614e..31036fe8c 100644 --- a/text/0017-pack-build-default-process-flag.md +++ b/text/0017-pack-build-default-process-flag.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Pack Build Default Process Flag - Start Date: 2019-10-29 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#28](https://github.com/buildpack/rfcs/pull/28) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0018-remove-pack-run.md b/text/0018-remove-pack-run.md index c250fb986..c8f506bc7 100644 --- a/text/0018-remove-pack-run.md +++ b/text/0018-remove-pack-run.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Remove `pack run` - Start Date: 2019-12-11 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0019-project-descriptor.md b/text/0019-project-descriptor.md index 8f05b534c..f3bc5cdf7 100644 --- a/text/0019-project-descriptor.md +++ b/text/0019-project-descriptor.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Minimal Project Descriptor - Start Date: 2019-06-11 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: https://github.com/buildpack/rfcs/pull/25 diff --git a/text/0020-landing-page.md b/text/0020-landing-page.md index b0575319b..e7a310d9f 100644 --- a/text/0020-landing-page.md +++ b/text/0020-landing-page.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Create a community landing page - Start Date: 1-6-2020 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#42](https://github.com/buildpack/rfcs/pull/42) - CNB Issue: 42 - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0021-lifecycle-compat-verification.md b/text/0021-lifecycle-compat-verification.md index 6e16fe253..2f8b4c3a8 100644 --- a/text/0021-lifecycle-compat-verification.md +++ b/text/0021-lifecycle-compat-verification.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Lifecycle Compatibility Verification - Start Date: 2019-12-06 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#34](https://github.com/buildpacks/rfcs/pull/34) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0023-circleci-orb.md b/text/0023-circleci-orb.md index 92b529abe..7f732fb73 100644 --- a/text/0023-circleci-orb.md +++ b/text/0023-circleci-orb.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Support a CircleCI Orb - Start Date: 2019-12-15 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#39](https://github.com/buildpacks/rfcs/pull/39) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0024-lifecycle-multicall-binary-for-build.md b/text/0024-lifecycle-multicall-binary-for-build.md index 355af9ded..40e1707ad 100644 --- a/text/0024-lifecycle-multicall-binary-for-build.md +++ b/text/0024-lifecycle-multicall-binary-for-build.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Lifecycle as a multicall binary for build phases - Start Date: 01/21/2020 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#45](https://github.com/buildpacks/rfcs/pull/45) - CNB Issue: (leave blank) - Supersedes: "N/A" diff --git a/text/0025-dont-trust-builders.md b/text/0025-dont-trust-builders.md index 925965a47..aefa1a5fb 100644 --- a/text/0025-dont-trust-builders.md +++ b/text/0025-dont-trust-builders.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Isolate Registry Credentials from Builder Images - Start Date: 2020-01-14 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#43](https://github.com/buildpacks/rfcs/pull/43) - CNB Issue: - Supersedes: N/A diff --git a/text/0026-lifecycle-all.md b/text/0026-lifecycle-all.md index c24ab0c0c..adc264cbb 100644 --- a/text/0026-lifecycle-all.md +++ b/text/0026-lifecycle-all.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Lifecycle All Binary - Start Date: 2020-01-21 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#46](https://github.com/buildpacks/rfcs/pull/46) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0027-spec-api-branches.md b/text/0027-spec-api-branches.md index 5b320b638..5b6fb5ed0 100644 --- a/text/0027-spec-api-branches.md +++ b/text/0027-spec-api-branches.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Spec API branches - Start Date: 2020-01-27 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#49](https://github.com/buildpacks/rfcs/pull/49) - CNB Issue: - Supersedes: N/A diff --git a/text/0028-remove-cred-helpers.md b/text/0028-remove-cred-helpers.md index 001cd624b..56e57a6d0 100644 --- a/text/0028-remove-cred-helpers.md +++ b/text/0028-remove-cred-helpers.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Remove Lifecycle Credential Helpers Integration - Start Date: 2020-01-22 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#49](https://github.com/buildpacks/rfcs/pull/49) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0029-template-changes.md b/text/0029-template-changes.md index cf245ef59..5c058bdf5 100644 --- a/text/0029-template-changes.md +++ b/text/0029-template-changes.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: RFC template changes - Start Date: 2020-02-03 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#52](https://github.com/buildpacks/rfcs/pull/52) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0030-links-for-buildpacks.md b/text/0030-links-for-buildpacks.md index 7774fb2aa..161f7d7bf 100644 --- a/text/0030-links-for-buildpacks.md +++ b/text/0030-links-for-buildpacks.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Buildpack homepage - Start Date: 1-23-2020 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#55](https://github.com/buildpacks/rfcs/pull/55) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0031-bionic-mixins.md b/text/0031-bionic-mixins.md index 9952fa8f7..899e03a59 100644 --- a/text/0031-bionic-mixins.md +++ b/text/0031-bionic-mixins.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Defining Mixins for io.buildpacks.stacks.bionic - Start Date: 2019-12-05 -- Status: Approved +- Status: Superseded - CNB Pull Request: https://github.com/buildpacks/rfcs/pull/40 - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0032-update-json-cnb-registry.md b/text/0032-update-json-cnb-registry.md index 72aaf2086..f14a4deda 100644 --- a/text/0032-update-json-cnb-registry.md +++ b/text/0032-update-json-cnb-registry.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Update JSON Structure of CNB Registry - Start Date: 2020-03-19 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: [RFC-0022](https://github.com/buildpacks/rfcs/blob/main/text/0022-client-side-buildpack-registry.md) diff --git a/text/0033-add-author.md b/text/0033-add-author.md index 54ef37ba7..df42b0517 100644 --- a/text/0033-add-author.md +++ b/text/0033-add-author.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Add Author to RFC Metadata - Start Date: 2020-03-14 -- Status: Approved +- Status: Implemented - CNB Pull Request: https://github.com/buildpacks/rfcs/pull/64 - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0034-image-labels.md b/text/0034-image-labels.md index 91e7788de..286e1fcf2 100644 --- a/text/0034-image-labels.md +++ b/text/0034-image-labels.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Image Labels - Start Date: 2020-03-24 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0035-buildpack-versions.md b/text/0035-buildpack-versions.md index 98024a3a0..dd8230c11 100644 --- a/text/0035-buildpack-versions.md +++ b/text/0035-buildpack-versions.md @@ -2,7 +2,7 @@ [meta]: #meta - **Name:** Semantic Version Formats for Buildpack Versions - **Start Date:** 2020-03-11 -- **Status:** Approved +- **Status:** Implemented - **Supersedes:** N/A # Summary diff --git a/text/0036-cnb-buildpack-directory-env-var.md b/text/0036-cnb-buildpack-directory-env-var.md index 1b0b78d92..67808c055 100644 --- a/text/0036-cnb-buildpack-directory-env-var.md +++ b/text/0036-cnb-buildpack-directory-env-var.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: CNB\_BUILDPACK\_DIR environment variable - Start Date: 2020-04-02 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#49](https://github.com/buildpacks/rfcs/pull/71) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0037-buildpack-uris.md b/text/0037-buildpack-uris.md index 616c759fd..12e259263 100644 --- a/text/0037-buildpack-uris.md +++ b/text/0037-buildpack-uris.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Buildpack URIs - Start Date: 02/07/2020 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#56](https://github.com/buildpacks/rfcs/pull/56) - CNB Issue: (leave blank) - Supersedes: "N/A" diff --git a/text/0038-gh-repo-labels.md b/text/0038-gh-repo-labels.md index 5fe9160a4..00de412e1 100644 --- a/text/0038-gh-repo-labels.md +++ b/text/0038-gh-repo-labels.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: GitHub repo labels - Start Date: 02/03/2020 -- Status: Approved +- Status: Implemented - CNB Pull Request: [rfcs#53](https://github.com/buildpacks/rfcs/pull/53) - CNB Issue: (leave blank) - Supersedes: N/A diff --git a/text/0039-release-process.md b/text/0039-release-process.md index 931a643e6..f9a83930f 100644 --- a/text/0039-release-process.md +++ b/text/0039-release-process.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Release Process - Start Date: 2019-11-18 -- Status: Approved +- Status: Implemented - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) diff --git a/text/0041-api-version-compat.md b/text/0041-api-version-compat.md index ad00d6a98..ae814167f 100644 --- a/text/0041-api-version-compat.md +++ b/text/0041-api-version-compat.md @@ -3,7 +3,7 @@ - Name: Lifecycle API version support changes - Start Date: 2020-05-14 - Author(s): [Emily Casey](https://github.com/ekcasey) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#79](https://github.com/buildpacks/rfcs/pull/79) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0042-process-specific-env.md b/text/0042-process-specific-env.md index 297a71276..a98e6d2bc 100644 --- a/text/0042-process-specific-env.md +++ b/text/0042-process-specific-env.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Process Specific Environment - Start Date: 2020-04-03 -- Status: Approved +- Status: Implemented - CNB Pull Request: (https://github.com/buildpacks/rfcs/pull/72) - CNB Issue: - Supersedes: N/A diff --git a/text/0043-increase-build-plan-flexibility.md b/text/0043-increase-build-plan-flexibility.md index eb51238b0..f4aa8cd46 100644 --- a/text/0043-increase-build-plan-flexibility.md +++ b/text/0043-increase-build-plan-flexibility.md @@ -3,7 +3,7 @@ - Name: Increase Build Plan Flexibility - Start Date: 2020-05-28 - Author(s): nebhale -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#82](https://github.com/buildpacks/rfcs/pull/82) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0044-pack-publish-buildpack.md b/text/0044-pack-publish-buildpack.md index dc0c8ec53..edeb03b0d 100644 --- a/text/0044-pack-publish-buildpack.md +++ b/text/0044-pack-publish-buildpack.md @@ -3,7 +3,7 @@ - Name: Pack Register Buildpack - Start Date: 2020-04-27 - Author(s): [Joe Kutner](https://github.com/jkutner), [Javier Romero](https://github.com/jromero) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#75](https://github.com/buildpacks/rfcs/pull/75) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0046-pack-pull-policy.md b/text/0046-pack-pull-policy.md index f6e7bd9d9..4b4b76226 100644 --- a/text/0046-pack-pull-policy.md +++ b/text/0046-pack-pull-policy.md @@ -3,7 +3,7 @@ - Name: `pack` Image Pull Policy - Start Date: 2020-05-21 - Author(s): [Javier Romero](https://github.com/jromero) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#80](https://github.com/buildpacks/rfcs/pull/80) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0047-danger-zone.md b/text/0047-danger-zone.md index 5188ede4a..a96e8bc9c 100644 --- a/text/0047-danger-zone.md +++ b/text/0047-danger-zone.md @@ -3,7 +3,7 @@ - Name: Read-Write Volume Mount in Pack - Start Date: 2020-06-02 - Author(s): nebhale -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#85](https://github.com/buildpacks/rfcs/pull/85) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0048-inline-buildpack.md b/text/0048-inline-buildpack.md index b8557d183..8a1da4791 100644 --- a/text/0048-inline-buildpack.md +++ b/text/0048-inline-buildpack.md @@ -3,7 +3,7 @@ - Name: Inline Buildpacks - Start Date: 2020-06-11 - Author(s): [Joe Kutner](https://github.com/jkutner) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#86](https://github.com/buildpacks/rfcs/pull/86) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0050-stack-metadata.md b/text/0050-stack-metadata.md index 3adbe8bed..52f40cf16 100644 --- a/text/0050-stack-metadata.md +++ b/text/0050-stack-metadata.md @@ -3,7 +3,7 @@ - Name: Additional Stack Metadata - Start Date: 2020-05-12 - Author(s): kvedurmu -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#78](https://github.com/buildpacks/rfcs/pull/78) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0051-override-env-by-default.md b/text/0051-override-env-by-default.md index 82d22eded..bf525f5d0 100644 --- a/text/0051-override-env-by-default.md +++ b/text/0051-override-env-by-default.md @@ -3,7 +3,7 @@ - Name: Override Env Vars by Default - Start Date: 2020-07-26 - Author(s): @sclevine -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#98](https://github.com/buildpacks/rfcs/pull/98) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0052-opt-in-layer-caching.md b/text/0052-opt-in-layer-caching.md index b54763649..957e54365 100644 --- a/text/0052-opt-in-layer-caching.md +++ b/text/0052-opt-in-layer-caching.md @@ -3,7 +3,7 @@ - Name: Opt-in Layer Caching - Start Date: 2020-07-26 - Author(s): @sclevine -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#99](https://github.com/buildpacks/rfcs/pull/99) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0054-project-descriptor-schema.md b/text/0054-project-descriptor-schema.md index a8d094769..7829c787c 100644 --- a/text/0054-project-descriptor-schema.md +++ b/text/0054-project-descriptor-schema.md @@ -3,7 +3,7 @@ - Name: Project Descriptor (project.toml) Schema - Start Date: 2020-28-07 - Author(s): @joshwlewis -- Status: Approved +- Status: Implemented - RFC Pull Request: - CNB Pull Request: - CNB Issue: diff --git a/text/0055-deprecate-service-bindings.md b/text/0055-deprecate-service-bindings.md index d9be209a8..ce2d5d00e 100644 --- a/text/0055-deprecate-service-bindings.md +++ b/text/0055-deprecate-service-bindings.md @@ -3,7 +3,7 @@ - Name: Deprecate Bindings Extension Specification - Start Date: 2020-08-06 - Author(s): @nebhale -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#105](https://github.com/buildpacks/rfcs/pull/105) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0056-any-stack-buildpacks.md b/text/0056-any-stack-buildpacks.md index 67b2eab53..4d389140b 100644 --- a/text/0056-any-stack-buildpacks.md +++ b/text/0056-any-stack-buildpacks.md @@ -3,7 +3,7 @@ - Name: Stackless Buildpacks - Start Date: 2020-07-26 - Author(s): @sclevine -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#97](https://github.com/buildpacks/rfcs/pull/97) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0057-exec.d-shell-free-profile-d.md b/text/0057-exec.d-shell-free-profile-d.md index b250c27bc..aa7c062ad 100644 --- a/text/0057-exec.d-shell-free-profile-d.md +++ b/text/0057-exec.d-shell-free-profile-d.md @@ -3,7 +3,7 @@ - Name: Exec.d - Shell-Free Profile.d - Start Date: 2020-07-26 - Author(s): @sclevine -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#104](https://github.com/buildpacks/rfcs/pull/104) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0058-pack-subcommands.md b/text/0058-pack-subcommands.md index 95544d32c..3b81017be 100644 --- a/text/0058-pack-subcommands.md +++ b/text/0058-pack-subcommands.md @@ -3,7 +3,7 @@ - Name: `pack` subcommands - Start Date: 2020-07-08 - Author(s): @jromero -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#93](https://github.com/buildpacks/rfcs/pull/93) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0059-label-rfcs.md b/text/0059-label-rfcs.md index 8c70203cf..39d6f3903 100644 --- a/text/0059-label-rfcs.md +++ b/text/0059-label-rfcs.md @@ -3,7 +3,7 @@ - Name: Label RFCs With Specification And Target Audience - Start Date: 2020-08-15 - Author(s): ForestEckhardt -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#108](https://github.com/buildpacks/rfcs/pull/108) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0060-create-repo-issues.md b/text/0060-create-repo-issues.md index 4791f2b10..92087a85d 100644 --- a/text/0060-create-repo-issues.md +++ b/text/0060-create-repo-issues.md @@ -3,7 +3,7 @@ - Name: Create Repo Issues - Start Date: 2020-08-06 - Author(s): [@dfreilich][https://github.com/dfreilich] -- Status: Approved +- Status: Superseded - RFC Pull Request: [rfcs#106](https://github.com/buildpacks/rfcs/pull/106) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0062-distribution-team.md b/text/0062-distribution-team.md index 0043b4d5e..278df08db 100644 --- a/text/0062-distribution-team.md +++ b/text/0062-distribution-team.md @@ -3,7 +3,7 @@ - Name: Create a Distribution Team - Start Date: 2020-09-03 - Author(s): [@jkutner](@jkutner) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#113](https://github.com/buildpacks/rfcs/pull/113) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/community#35](https://github.com/buildpacks/community/issues/35) diff --git a/text/0063-stack-images-config-env-path-must-exist.md b/text/0063-stack-images-config-env-path-must-exist.md index abbbf84b0..7693315c0 100644 --- a/text/0063-stack-images-config-env-path-must-exist.md +++ b/text/0063-stack-images-config-env-path-must-exist.md @@ -3,7 +3,7 @@ - Name: Build/Run image configs MUST contain env.PATH - Start Date: 2020-08-21 - Author(s): @aemengo @ameyer-pivotal @mvalliath @TisVictress @micahyoung -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#114](https://github.com/buildpacks/rfcs/pull/114) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#150](https://github.com/buildpacks/spec/issues/150), [buildpacks/docs#223](https://github.com/buildpacks/docs/issues/223) diff --git a/text/0064-buildpacks-contribute-default-process-type.md b/text/0064-buildpacks-contribute-default-process-type.md index 3c28a145e..3466a605d 100644 --- a/text/0064-buildpacks-contribute-default-process-type.md +++ b/text/0064-buildpacks-contribute-default-process-type.md @@ -3,7 +3,7 @@ - Name: Buildpacks should be able to define the default process type for an app - Start Date: 8/20/20 - Author(s): natalieparellano -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#110](https://github.com/buildpacks/rfcs/pull/110) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#159](https://github.com/buildpacks/spec/issues/159), [buildpacks/spec#160](https://github.com/buildpacks/spec/issues/160), [buildpacks/lifecycle#457](https://github.com/buildpacks/lifecycle/issues/457), [buildpacks/lifecycle#458](https://github.com/buildpacks/lifecycle/issues/458), [buildpacks/pack#946](https://github.com/buildpacks/pack/issues/946), [buildpacks/docs#246](https://github.com/buildpacks/docs/issues/246) diff --git a/text/0067-report-toml-image-manifest-size.md b/text/0067-report-toml-image-manifest-size.md index 24f7cfe6c..0f734fee0 100644 --- a/text/0067-report-toml-image-manifest-size.md +++ b/text/0067-report-toml-image-manifest-size.md @@ -3,7 +3,7 @@ - Name: Add Image Manifest Size to report.toml - Start Date: 2020-11-02 - Author(s): djoyahoy -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#121](https://github.com/buildpacks/rfcs/pull/121) - CNB Pull Request: - CNB Issue: [buildpacks/spec#169](https://github.com/buildpacks/spec/issues/169), [buildpacks/lifecycle#490](https://github.com/buildpacks/lifecycle/issues/490) diff --git a/text/0068-buildpack-registry-search-api.md b/text/0068-buildpack-registry-search-api.md index 40e97b6eb..aa67b1955 100644 --- a/text/0068-buildpack-registry-search-api.md +++ b/text/0068-buildpack-registry-search-api.md @@ -3,7 +3,7 @@ - Name: Buildpack Registry Search API - Start Date: 2020-11-11 - Author(s): @elbandito -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#125](https://github.com/buildpacks/rfcs/pull/125) - CNB Pull Request: (leave blank) - CNB Issue: [registry-api#3](https://github.com/buildpacks/registry-api/issues/3) diff --git a/text/0070-new-buildpack-toml-keys.md b/text/0070-new-buildpack-toml-keys.md index 5661433d9..62287f05f 100644 --- a/text/0070-new-buildpack-toml-keys.md +++ b/text/0070-new-buildpack-toml-keys.md @@ -3,7 +3,7 @@ - Name: New Buildpack Descriptor Keys - Start Date: 2020-12-12 - Author(s): [@jkutner](@jkutner) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#127](https://github.com/buildpacks/rfcs/pull/127) - CNB Pull Request: [buildpacks/pack#1022](https://github.com/buildpacks/pack/pull/1022) - CNB Issue: [buildpacks/spec#181](https://github.com/buildpacks/spec/issues/181), [buildpacks/docs#288](https://github.com/buildpacks/docs/issues/288) diff --git a/text/0071-cnb-user-research-2021.md b/text/0071-cnb-user-research-2021.md index bc9013a33..c0efc8ba8 100644 --- a/text/0071-cnb-user-research-2021.md +++ b/text/0071-cnb-user-research-2021.md @@ -3,7 +3,7 @@ - Name: CNB User Research 2021 - Start Date: 2020-12-10 - Author(s): [@sampeinado](@sampeinado) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#126](https://github.com/buildpacks/rfcs/pull/126) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) diff --git a/text/0072-image-workdir.md b/text/0072-image-workdir.md index 2a15c136e..b81b4f318 100644 --- a/text/0072-image-workdir.md +++ b/text/0072-image-workdir.md @@ -3,7 +3,7 @@ - Name: Set Image Working Directory to value of CNB_APP_DIR - Start Date: 2021-01-13 - Author(s): @josegonzalez -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#134](https://github.com/buildpacks/rfcs/pull/134) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#183](https://github.com/buildpacks/spec/issues/183), [buildpacks/lifecycle#516](https://github.com/buildpacks/lifecycle/issues/516), [buildpacks/docs#290](https://github.com/buildpacks/docs/issues/290) diff --git a/text/0074-layer-table.md b/text/0074-layer-table.md index 14d9bfc26..3adcc0123 100644 --- a/text/0074-layer-table.md +++ b/text/0074-layer-table.md @@ -3,7 +3,7 @@ - Name: Move layer types to new table in layer.toml - Start Date: 2021-01-05 - Author(s): natalieparellano -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#132](https://github.com/buildpacks/rfcs/pull/132) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#185](https://github.com/buildpacks/spec/issues/185), [buildpacks/lifecycle#522](https://github.com/buildpacks/lifecycle/issues/522), [buildpacks/docs#296](https://github.com/buildpacks/docs/issues/296) diff --git a/text/0075-move-analyze-phase.md b/text/0075-move-analyze-phase.md index ee87bdc5e..3724619b7 100644 --- a/text/0075-move-analyze-phase.md +++ b/text/0075-move-analyze-phase.md @@ -3,7 +3,7 @@ - Name: Move analyze phase - Start Date: 2021-01-13 - Author(s): [@jkutner](github.com/jkutner/) [@jabrown85](github.com/jabrown85) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#135](https://github.com/buildpacks/rfcs/pull/135) - CNB Pull Request: [buildpacks/spec#172](https://github.com/buildpacks/spec/pull/172) - CNB Issue: [buildpacks/spec#194](https://github.com/buildpacks/spec/issues/194), [buildpacks/lifecycle#530](https://github.com/buildpacks/lifecycle/issues/530), [buildpacks/pack#1078](https://github.com/buildpacks/pack/issues/1078), [buildpacks/docs#302](https://github.com/buildpacks/docs/issues/302) diff --git a/text/0077-pack-buildpack-create.md b/text/0077-pack-buildpack-create.md index 850ea1485..f180da574 100644 --- a/text/0077-pack-buildpack-create.md +++ b/text/0077-pack-buildpack-create.md @@ -3,7 +3,7 @@ - Name: Pack Command to Create a Buildpack Repo - Start Date: 2021-01-19 - Author(s): [jkutner](https://github.com/jkutner) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#136](https://github.com/buildpacks/rfcs/pull/136) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1025](https://github.com/buildpacks/pack/issues/1025) diff --git a/text/0078-group-additions.md b/text/0078-group-additions.md index 9eed25872..df01c6354 100644 --- a/text/0078-group-additions.md +++ b/text/0078-group-additions.md @@ -3,7 +3,7 @@ - Name: Group additions to Builder order - Start Date: 2020-12-23 - Author(s): [jkutner](@jkutner) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#129](https://github.com/buildpacks/rfcs/pull/129) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/docs#319](https://github.com/buildpacks/docs/issues/319), [buildpacks/pack#1099](https://github.com/buildpacks/pack/issues/1099), [buildpacks/pack#1100](https://github.com/buildpacks/pack/issues/1100), [buildpacks/spec#195](https://github.com/buildpacks/spec/issues/195) diff --git a/text/0080-builder-key-in-project-descriptor.md b/text/0080-builder-key-in-project-descriptor.md index 293842ef9..a262cee9d 100644 --- a/text/0080-builder-key-in-project-descriptor.md +++ b/text/0080-builder-key-in-project-descriptor.md @@ -3,7 +3,7 @@ - Name: Builder key in project descriptor - Start Date: 2021-02-11 - Author(s): @wburningham -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#138](https://github.com/buildpacks/rfcs/pull/138) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#215](https://github.com/buildpacks/spec/issues/215), [buildpacks/pack#1114](https://github.com/buildpacks/pack/issues/1114), [buildpacks/docs#345](https://github.com/buildpacks/docs/issues/345) diff --git a/text/0081-process-specific-working-directory.md b/text/0081-process-specific-working-directory.md index 370df0e1d..60ccd1ae3 100644 --- a/text/0081-process-specific-working-directory.md +++ b/text/0081-process-specific-working-directory.md @@ -3,7 +3,7 @@ - Name: Process Specific Working Directory - Start Date: 2021-03-09 - Author(s): ForestEckhardt, ryanmoran, fg-j -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#144](https://github.com/buildpacks/rfcs/pull/144) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#212](https://github.com/buildpacks/spec/issues/212), [buildpacks/spec#216](https://github.com/buildpacks/spec/issues/216) diff --git a/text/0082-pack-package-refactor.md b/text/0082-pack-package-refactor.md index 54bc382a9..2a67c4a8d 100644 --- a/text/0082-pack-package-refactor.md +++ b/text/0082-pack-package-refactor.md @@ -3,7 +3,7 @@ - Name: Refactor Pack go packages - Start Date: 02/24/21 - Author(s): dwillist -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#139](https://github.com/buildpacks/rfcs/pull/139) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1128](https://github.com/buildpacks/pack/issues/1128) diff --git a/text/0084-project-descriptor-domains.md b/text/0084-project-descriptor-domains.md index bb3077fe0..94cdd65cc 100644 --- a/text/0084-project-descriptor-domains.md +++ b/text/0084-project-descriptor-domains.md @@ -3,7 +3,7 @@ - Name: Project Descriptor Domains - Start Date: 2021-02-26 - Author(s): hone -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#140](https://github.com/buildpacks/rfcs/pull/140) - CNB Pull Request: (leave blank) - CNB Issue: N/A diff --git a/text/0085-run-uid.md b/text/0085-run-uid.md index dd6c89060..3b2ae4237 100644 --- a/text/0085-run-uid.md +++ b/text/0085-run-uid.md @@ -3,7 +3,7 @@ - Name: Recommending different run-time and build-time users - Start Date: 2021-03-15 - Author(s): [@samj1912](https://github.com/samj1912) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#146](https://github.com/buildpacks/rfcs/pull/146) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#223](https://github.com/buildpacks/spec/issues/223) diff --git a/text/0088-minimum-docs-lifecycle.md b/text/0088-minimum-docs-lifecycle.md index d07603b36..2d9ff6c24 100644 --- a/text/0088-minimum-docs-lifecycle.md +++ b/text/0088-minimum-docs-lifecycle.md @@ -3,7 +3,7 @@ - Name: Define a minimum standard for docs in order to ship the lifecycle - Start Date: 4/7/2021 - Author(s): natalieparellano -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#153](https://github.com/buildpacks/rfcs/pull/153) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/lifecycle#627](https://github.com/buildpacks/lifecycle/issues/627) diff --git a/text/0089-buildpack-authors-tooling-subteam.md b/text/0089-buildpack-authors-tooling-subteam.md index 61da5ed0c..887d2f1fe 100644 --- a/text/0089-buildpack-authors-tooling-subteam.md +++ b/text/0089-buildpack-authors-tooling-subteam.md @@ -3,7 +3,7 @@ - Name: Buildpack Authors' Tooling Sub-Team - Start Date: 2021-05-11 - Author(s): [@ekcasey](https://github.com/ekcasey), [@hone](https://github.com/hone), [@samj1912](https://github.com/samj1912) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#159](https://github.com/buildpacks/rfcs/pull/159) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/community#102](https://github.com/buildpacks/community/issues/102) diff --git a/text/0091-pack-cache-options.md b/text/0091-pack-cache-options.md index d4cdf2aaf..b62203af9 100644 --- a/text/0091-pack-cache-options.md +++ b/text/0091-pack-cache-options.md @@ -3,7 +3,7 @@ - Name: Pack cache options - Start Date: 2021-03-25 - Author(s): [@jromero](https://github.com/jromero), [@dwillist](https://github.com/dwillist) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#149](https://github.com/buildpacks/rfcs/pull/149) - CNB Pull Request: (leave blank) - CNB Issue: [pack#1077](https://github.com/buildpacks/pack/issues/1077) diff --git a/text/0092-add-visual-pack-build.md b/text/0092-add-visual-pack-build.md index b20e16657..8a56d5ee4 100644 --- a/text/0092-add-visual-pack-build.md +++ b/text/0092-add-visual-pack-build.md @@ -3,7 +3,7 @@ - Name: Add Visual Pack Build - Start Date: 2020-05-14 - Author(s): aemengo -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#160](https://github.com/buildpacks/rfcs/pull/160) - CNB Pull Request: (leave blank) - CNB Issue: [pack#1200](https://github.com/buildpacks/pack/issues/1200), [pack#1201](https://github.com/buildpacks/pack/issues/1201), [pack#1203](https://github.com/buildpacks/pack/issues/1203), [pack#1204](https://github.com/buildpacks/pack/issues/1204), [pack#1205](https://github.com/buildpacks/pack/issues/1205), [pack#1206](https://github.com/buildpacks/pack/issues/1206) diff --git a/text/0093-remove-shell-processes.md b/text/0093-remove-shell-processes.md index 0e9a88d50..6f95074b0 100644 --- a/text/0093-remove-shell-processes.md +++ b/text/0093-remove-shell-processes.md @@ -3,7 +3,7 @@ - Name: Remove Shell Processes - Start Date: 2021-05-14 - Author(s): @ekcasey -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#168](https://github.com/buildpacks/rfcs/pull/168) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/spec#244](https://github.com/buildpacks/spec/issues/244), [buildpacks/spec#245](https://github.com/buildpacks/spec/issues/245), [buildpacks/lifecycle#693](https://github.com/buildpacks/lifecycle/issues/693), [buildpacks/pack#1260](https://github.com/buildpacks/pack/issues/1260), [buildpacks/docs#391](https://github.com/buildpacks/docs/issues/391), [buildpacks/libcnb#70](https://github.com/buildpacks/libcnb/issues/70) diff --git a/text/0094-add-status-field.md b/text/0094-add-status-field.md index 5dd946abb..b3c9162ad 100644 --- a/text/0094-add-status-field.md +++ b/text/0094-add-status-field.md @@ -3,7 +3,7 @@ - Name: Add Status to RFC Metadata - Start Date: 2021-07-14 - Author(s): @aemengo, @ekcasey -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#177](https://github.com/buildpacks/rfcs/pull/177) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/rfcs#183](#https://github.com/buildpacks/rfcs/issues/183) diff --git a/text/0095-sbom.md b/text/0095-sbom.md index 7c7ae04be..740f81a5b 100644 --- a/text/0095-sbom.md +++ b/text/0095-sbom.md @@ -3,7 +3,7 @@ - Name: Structured SBOMs - Start Date: 2021-06-02 - Author(s): [@samj1912](https://github.com/samj1912), [@sophiewigmore](https://github.com/sophiewigmore), [@ForestEckhardt](https://github.com/ForestEckhardt) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#166](https://github.com/buildpacks/rfcs/pull/166) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/lifecycle#732](https://github.com/buildpacks/lifecycle/issues/732), [buildpacks/lifecycle#733](https://github.com/buildpacks/lifecycle/issues/733), [buildpacks/lifecycle#734](https://github.com/buildpacks/lifecycle/issues/734), [buildpacks/lifecycle#735](https://github.com/buildpacks/lifecycle/issues/735), [buildpacks/lifecycle#736](https://github.com/buildpacks/lifecycle/issues/736), [buildpacks/lifecycle#737](https://github.com/buildpacks/lifecycle/issues/737), [buildpacks/lifecycle#738](https://github.com/buildpacks/lifecycle/issues/738) diff --git a/text/0097-official-utility-buildpacks.md b/text/0097-official-utility-buildpacks.md index 4e7477090..e5e6beb75 100644 --- a/text/0097-official-utility-buildpacks.md +++ b/text/0097-official-utility-buildpacks.md @@ -7,6 +7,7 @@ - CNB Pull Request: (leave blank) - CNB Issue: N/A - Supersedes: N/A +- Status: Implemented # Summary [summary]: #summary diff --git a/text/0098-rfc-issue-generation.md b/text/0098-rfc-issue-generation.md index 8d112829f..4effb3978 100644 --- a/text/0098-rfc-issue-generation.md +++ b/text/0098-rfc-issue-generation.md @@ -3,7 +3,7 @@ - Name: RFC Issue Generation - Start Date: 2020-03-03 - Author(s): @jromero -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#141](https://github.com/buildpacks/rfcs/pull/141) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/rfcs#158](https://github.com/buildpacks/rfcs/issues/158) diff --git a/text/0099-slack.md b/text/0099-slack.md index a0ece6a29..a9e3fb71f 100644 --- a/text/0099-slack.md +++ b/text/0099-slack.md @@ -3,7 +3,7 @@ - Name: Migrate to CNCF Slack - Start Date: 2021-12-07 - Author(s): [@samj1912](https://github.com/samj1912) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#198](https://github.com/buildpacks/rfcs/pull/198) - CNB Pull Request: (leave blank) - CNB Issue: N/A diff --git a/text/0100-buildpack-input-vars.md b/text/0100-buildpack-input-vars.md index 27782d1dc..ee1dee3aa 100644 --- a/text/0100-buildpack-input-vars.md +++ b/text/0100-buildpack-input-vars.md @@ -3,7 +3,7 @@ - Name: Replace positional args to Buildpack executables with env vars - Start Date: 2021-11-18 - Author(s): [jkutner](https://github.com/jkutner) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#190](https://github.com/buildpacks/rfcs/pull/190) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/lifecycle#806](https://github.com/buildpacks/lifecycle/issues/806), [buildpacks/lifecycle#807](https://github.com/buildpacks/lifecycle/issues/807) diff --git a/text/0101-system-buildpacks-in-builder.md b/text/0101-system-buildpacks-in-builder.md index 18a912404..555bc9e57 100644 --- a/text/0101-system-buildpacks-in-builder.md +++ b/text/0101-system-buildpacks-in-builder.md @@ -7,6 +7,7 @@ - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/rfcs#209](https://github.com/buildpacks/rfcs/issues/209) - Supersedes: N/A +- Status: Approved # Summary [summary]: #summary diff --git a/text/0103-source-date-epoch.md b/text/0103-source-date-epoch.md index 284fe997f..6d55c386a 100644 --- a/text/0103-source-date-epoch.md +++ b/text/0103-source-date-epoch.md @@ -3,7 +3,7 @@ - Name: Use SOURCE_DATE_EPOCH to configure image create time - Start Date: 2022-02-24 - Author(s): natalieparellano -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#204](https://github.com/buildpacks/rfcs/pull/204) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/lifecycle#809](https://github.com/buildpacks/lifecycle/issues/809) diff --git a/text/0104-dot-profile-utility-buildpack.md b/text/0104-dot-profile-utility-buildpack.md index bdd60c2e5..be5f6bea5 100644 --- a/text/0104-dot-profile-utility-buildpack.md +++ b/text/0104-dot-profile-utility-buildpack.md @@ -3,7 +3,7 @@ - Name: `.profile` Utility Buildpack - Start Date: 2022-01-25 - Author(s): mboldt -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#200](https://github.com/buildpacks/rfcs/pull/200) - CNB Pull Request: (leave blank) - CNB Issue: N/A diff --git a/text/0105-dockerfiles.md b/text/0105-dockerfiles.md index ceb98f394..6af6023a7 100644 --- a/text/0105-dockerfiles.md +++ b/text/0105-dockerfiles.md @@ -8,6 +8,7 @@ - CNB Issue: [buildpacks/lifecycle#890](https://github.com/buildpacks/lifecycle/issues/890), [buildpacks/lifecycle#891](https://github.com/buildpacks/lifecycle/issues/891), [buildpacks/lifecycle#892](https://github.com/buildpacks/lifecycle/issues/892), [buildpacks/lifecycle#893](https://github.com/buildpacks/lifecycle/issues/893), [buildpacks/lifecycle#894](https://github.com/buildpacks/lifecycle/issues/894) - Supersedes: [RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md), [RFC#167](https://github.com/buildpacks/rfcs/pull/167) - Depends on: [RFC#172](https://github.com/buildpacks/rfcs/pull/172) +- Status: Approved # Summary [summary]: #summary diff --git a/text/0106-rfc-process.md b/text/0106-rfc-process.md index 6f60ead6a..7b36509fa 100644 --- a/text/0106-rfc-process.md +++ b/text/0106-rfc-process.md @@ -3,7 +3,7 @@ - Name: RFC Process - Start Date: 2021-05-10 - Author(s): [@ekcasey](https://github.com/ekcasey) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#233](https://github.com/buildpacks/rfcs/pull/233) - CNB Pull Request: (leave blank) - CNB Issue: N/A From f6db602291e0becc4caa3ce5e0161e2191dd5138 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Fri, 2 Dec 2022 09:41:01 -0600 Subject: [PATCH 221/372] Finalized 0113-additonal-oci-artifacts Signed-off-by: Joe Kutner --- ...cts.md => 0113-additonal-oci-artifacts.md} | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) rename text/{0000-additonal-oci-artifacts.md => 0113-additonal-oci-artifacts.md} (88%) diff --git a/text/0000-additonal-oci-artifacts.md b/text/0113-additonal-oci-artifacts.md similarity index 88% rename from text/0000-additonal-oci-artifacts.md rename to text/0113-additonal-oci-artifacts.md index a4f3b7593..0954b032a 100644 --- a/text/0000-additonal-oci-artifacts.md +++ b/text/0113-additonal-oci-artifacts.md @@ -3,45 +3,45 @@ - Name: Additional OCI Artifacts from Buildpacks - Start Date: 2022-04-24 - Author(s): agracey -- Status: Draft -- RFC Pull Request: (leave blank) -- CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) +- Status: Approved +- RFC Pull Request: https://github.com/buildpacks/rfcs/pull/223 +- CNB Pull Request: n/a +- CNB Issue: n/a +- Supersedes: n/a # Summary [summary]: #summary -Allow buildpack authors to specify additional artifacts from their buildpacks to be uploaded to a registry along with the standard set of layers. +Allow buildpack authors to specify additional artifacts from their buildpacks to be uploaded to a registry along with the standard set of layers. # Definitions [definitions]: #definitions - WebAssembly (WASM) -- Portable binaries to be run with a WebAssembly virtual machine. -- [OCI Artifact](https://github.com/opencontainers/artifacts/blob/main/definitions-terms.md#media-type) -- Any blob of data that can be stored in an OCI registry. From the spec: +- [OCI Artifact](https://github.com/opencontainers/artifacts/blob/main/definitions-terms.md#media-type) -- Any blob of data that can be stored in an OCI registry. From the spec: > An artifact has a type, which has a collection of layers. The Artifact is defined as unique by its `manifest.config.mediaType`. Layers are defined by their `layer.config.mediaType`. # Motivation [motivation]: #motivation -Cloud Native Buildpacks are awesome for building container images but come with a draw back that they can only produce a single output. This is limiting to platform builders because there are many cases where you might want to have multiple outputs from the same build process. +Cloud Native Buildpacks are awesome for building container images but come with a draw back that they can only produce a single output. This is limiting to platform builders because there are many cases where you might want to have multiple outputs from the same build process. # What it is [what-it-is]: #what-it-is This change allows for additional flexibility in what buildpack authors and platforms can build on top of buildpacks. Three initial use cases are WebAssembly, test output, and Helm charts. -For the first, a buildpack would be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or runwasi). +For the first, a buildpack would be able to also generate a wasm bundle that can be used by a downstream platform (such as the Krustlet or runwasi). Similarly, by allowing the buildpack to produce a helm chart, you could integrate in logic about how to set up an application into the build pipeline where you are able to do bits of code analysis. (NetworkPolicies that lock down an app, sidecars that need to be run, ConfigMaps to pass around out-of-band, etc...) -Lastly, it may be advantageous to allow a buildpack to produce the results of unit tests and publish alongside the container itself. Then a tool (such as Kubewarden) could gate running of the container based on the passing tests. +Lastly, it may be advantageous to allow a buildpack to produce the results of unit tests and publish alongside the container itself. Then a tool (such as Kubewarden) could gate running of the container based on the passing tests. # How it Works [how-it-works]: #how-it-works -Add an optional file per layer called `artifacts.toml` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. +Add an optional file per layer called `artifacts.toml` that tells lifecycle to publish the specified file(s) in the layer to an OCI registry in the publish step. It also needs to allow for setting labels and the tag to publish with. For the WASM use-case, a buildpack author looking to also publish a node.js WASM bundle would write the build logic like any other buildpack. Create a directory in $LAYERS_DIR with the following `artifacts.toml`: @@ -60,17 +60,17 @@ As a note: to keep this generic to any potential OCI artifact, it seems best to # Migration [migration]: #migration -This functionality should be purely additive and not break existing buildpacks. +This functionality should be purely additive and not break existing buildpacks. # Drawbacks [drawbacks]: #drawbacks -This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. This can be mitigated with good docs and platform output. Adding a change to make the standard image output optional seems like a larger change than is appropriate at the moment. +This does bring in additional scope to the project and could potentially lead to confusion around what OCI image that got published should be used. This can be mitigated with good docs and platform output. Adding a change to make the standard image output optional seems like a larger change than is appropriate at the moment. # Alternatives [alternatives]: #alternatives -Other ways of doing this would be to add the artifacts to the primary OCI image directly then pull back out in a future stage. This has two obvious issues: artifact size and escalation in tooling needed in pipeline. +Other ways of doing this would be to add the artifacts to the primary OCI image directly then pull back out in a future stage. This has two obvious issues: artifact size and escalation in tooling needed in pipeline. # Prior Art [prior-art]: #prior-art From 444e7d8066eb2cea57e9d3c2ea53009e12362e0e Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 2 Dec 2022 11:08:18 -0500 Subject: [PATCH 222/372] Tracking issue should prompt maintainers to update the RFC status. Signed-off-by: Natalie Arellano --- .github/ISSUE_TEMPLATE/tracking.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/tracking.md b/.github/ISSUE_TEMPLATE/tracking.md index e9e1d470e..045c7c379 100644 --- a/.github/ISSUE_TEMPLATE/tracking.md +++ b/.github/ISSUE_TEMPLATE/tracking.md @@ -31,3 +31,5 @@ Documentation: : - [ ] - [ ] Released in version `` + +**Maintainers**: when closing this issue as completed, submit a PR to update the `Status` of the RFC to `Implemented`. From 6700d84b92048952eb4e26d56d8648e45f63dbcb Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 2 Dec 2022 12:43:50 -0500 Subject: [PATCH 223/372] Remove references to custom env templating As this is a breaking change, we decided to do this in a separate (yet to be created) RFC. Created issue: https://github.com/buildpacks/rfcs/issues/258 In addition to the changes described originally in 0093 we'd like some way of versioning the launcher interface, to avoid surprising end-users. Signed-off-by: Natalie Arellano --- text/0093-remove-shell-processes.md | 110 ++-------------------------- 1 file changed, 6 insertions(+), 104 deletions(-) diff --git a/text/0093-remove-shell-processes.md b/text/0093-remove-shell-processes.md index 0e9a88d50..fd6d380da 100644 --- a/text/0093-remove-shell-processes.md +++ b/text/0093-remove-shell-processes.md @@ -88,15 +88,12 @@ The following changes have been made: - **`direct` has been removed** - all processes are executed directly. If `bash` or `cmd.exe` is required it should be included in `command`. No surprises. - **`command` is now an array** - Arguments in `command` *will not* be overwritten if a user provides additional arguments at runtime. `args` are default arguments that *will* be overwritten if a user provides additional arguments at runtime. The makes `command` analogous to `Entrypoint` in the OCI spec and `command` in a Kubernetes PodSpec. `args` is analogous to `cmd` and `args` in docker and Kubernetes respectively. -## Using Environment Variables in a Process - -One upside to our previous execution strategy was that it enable users to include environment variable references in arguments that were later evaluated in the container. To preserve this feature we can instead adopt the Kubernetes strategy for [environment variables interpolation](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config). If a buildpack or user includes `$()` in the `command` or `args` and `` is the name of an environment variable set in the launch environment, the launcher will replace this string with the value of the environment variable after apply buildpack-provided env modifications and before launching the process. - # How it Works [how-it-works]: #how-it-works ## Buildpack-provided process types + ### Example 1 - A Shell Process The Paketo .Net Execute Buildpack may generates shell processes similar to the following: ``` @@ -110,12 +107,11 @@ NOTE: the buildpack API used by this buildpack (`0.5`) predates the introduction Using the new API this process could look like: ``` [[processes]] -type = "web" -command = ["dotnet", "my-app.dll", "--urls", "http://0.0.0.0:$(PORT)"] # the default value of PORT would need to be provided in a layer +type = "bash" +command = ["-c", "dotnet", "my-app.dll", "--urls", "http://0.0.0.0:${PORT:-8080}"] default = true ``` Things to note: -* In the above example I have eliminated the dependency on Bash instead of explicitly adding it to the command, because it is likely unnecessary. * If the buildpack authors believed that `--urls` should be overridable they could set move the last two arguments from `command` to `args`. ### Example 2 - A Script Process @@ -139,39 +135,9 @@ command = ["bash", "-c", "pre-start.sh && nodejs server.js && post-start.sh"] default = false ``` -## User Provided Processes - -Currently if the user can specify a custom process dynamically at runtime by setting the container entrypoint to `launcher` directly rather than using a symlink to the launcher, the providing a custom `cmd`. This custom command is executed directly if `cmd` is an array and the first element is `--`. Otherwise the custom command is assumed to be a shell process. In the interest of removing complexity we should do away with the special `--` argument and execute all custom commands directly. +## User-provided process types -### Example 1 - A Direct process -The follow direct commands: -``` -docker run --entrypoint launcher -- env -docker run --entrypoint launcher -- echo hello '$WORLD' -``` -will become the following, using the new platform API -``` -docker run --entrypoint launcher env -docker run --entrypoint launcher echo hello '$(WORLD)' -``` -Previously, in the second command in this example, `$WORLD` would not have been interpolated because this is a direct process; instead the output would include the literal string `$WORLD`. With the changes proposed, `$(WORLD)` will now be evaluated, even though the process is direct. - -### Example 2 - A Shell Process -The follow custom shell command: -``` -docker run --entrypoint launcher echo hello '${WORLD}' -docker run --entrypoint launcher echo hello '${WORLD:-world}' -``` -will become the following, using the new platform API -``` -docker run --entrypoint launcher echo hello '$(WORLD)' -docker run --entrypoint launcher bash -c 'echo hello "${WORLD:-world}"' -``` -The first command in this example needed to adopt the new environment variable syntax to behave as expected with the new API. Previously it was necessary to use a shell process in order to evaluate `${WORLD}`. Now, the shell is unnecessary. - -If the user wishes, they may explicitly invoke a shell and let Bash handle the interpolation, which provides a richer feature set. - -### Example 3 - A Script Process +### Example 1 - A Script Process The follow custom script command: ``` docker run --entrypoint launcher 'for opt in $JAVA_OPTS; do echo $opt; done' @@ -181,35 +147,6 @@ will become the following, using the new platform API docker run --entrypoint launcher bash -c 'for opt in $JAVA_OPTS; do echo $opt; done' ``` -### Example 4 - A Script Process in Kubernetes - -Because we have adopted the Kubernetes environment variable notation here, users may need to escape some references in their PodSpec in specific situations. This is necessary only if all of the following are true: -* The user is providing a `command` or `args` which contain an environment variable reference. -* The variable is explicitly initialized in the `env` section of the PodSpec. -* The user wishes for the variable to be interpolated **after** build-provided env modifications have been applied. - -``` -apiVersion: v1 -kind: Pod -metadata: - name: env-example -spec: - containers: - - name: env-print-demo - image: bash - env: - - name: IN_CONTAINER_1 - value: "k8s-val" - - name: IN_K8S - value: "val2" - command: ["bash", "-c", "echo $$(IN_CONTAINER_1)) $(IN_CONTAINER_2) $(IN_K8S) ${IN_BASH}"] -``` -In the above example the environment variables will be interpolated as follows: -- `$IN_CONTAINER` - Interpolated by the launcher after buildpack-provided modifications (e.g. `k8s-val:buildpack-appended-val`) -- `$IN_CONTAINER_2` - Interpolated by the launcher after buildpack-provided modifications. No escaping is required here because `$IN_CONTAINER_2` is not set in `env`. -- `$IN_K8S` - Interpolated by Kubernetes before the container runs. Buildpack-provided modifications will not affect the resulting value. -- `$IN_BASH` - Interpolated by Bash. - ## What About Profile Scripts? ### `/profile.d/*` @@ -295,7 +232,6 @@ we would convert it to the following in `metadata.toml` type = "hi" command = ["echo", "hello", "world"] args = [] -api = "0.5" direct = true ``` This spares users from having to learn about the differences between buildpack APIs in order to predict how additional arguments will behave. @@ -319,7 +255,6 @@ It should be converted to the following in `metadata.toml` type = "hi" command = ["echo", "hello", "${WORLD:-world}"] args = [] -api = "0.5" direct = false ``` @@ -339,7 +274,6 @@ When migrating to the new API, buildpack authors should take the following steps 1. Does the buildpack contribute a `direct=false` process? If so, there are two options: 1. Explicitly add `bash` or `cmd` to the process definition, this is the safest path forward. 2. Remove an unnecessary dependency on `bash` or `cmd` by: - * Changing any environment variable references to use the `$())` syntax - -We could use our own explicit syntax like `$(())` or environment variable replacements instead of `$()`. - -pros: -* Extremely explicit -* There is never any need to escape values in k8s -* Eliminates all conflicts or situations where the evaluation order must be explained - -cons: -* A third syntax -* A buildpack-specific syntax that users must learn about through documentation - -## `${}` syntax -We could use Bash-like `${}` syntax for environment variable replacements instead of `$()`. - -pros: -* It might "just work" in a lot of cases -* Familiar - -cons: -* The launcher might evaluate env vars that were truly intended for Bash to evaluate (e.g. in a command like `["bash", "-c", "source foo-setter.sh && echo ${FOO}"]`) -* Users might reasonably expect things like `$FOO` or `${FOO:-dafault-foo}` to work and discover the limitations only via trial and error or documentation. - ## Lifecycle support for `/.exec` When we remove support for `/.profile` we could add support for `/.exec` or similar where `/.exec` must implement the `exec.d` interface. @@ -426,15 +336,7 @@ However, things get very complicated if we explicitly source profiles or shell e The Paketo Java buildpacks have already converted an extensive collection of profile script to exec.d binaries and converted all processes to direct processes in order to support minimal stacks like `io.paketo.stacks.tiny` and to provide users with more intuitive argument handling. -The Procfile interface is supported by Paketo, Heroku, and Google buildpacks, demonstrating that it is possible to have a consistent interface across buildpack ecosystems without building direct support for that interface in the lifecycle. - -# Unresolved Questions -[unresolved-questions]: #unresolved-questions - -Do we need to provide a mechanism for turning `$(env)` interpolation off? E.g. -``` -docker run --env "CNB_INTERPOLATE=false" --entrypoint launcher echo hello '${WORLD}' # prints "hello '$(WORLD)'" -``` +The Procfile interface is supported by Paketo, Heroku, and Google buildpacks, demonstrating that it is possible to have a consistent interface across buildpack ecosystems without building direct support for that interface in the lifecycle. # Spec. Changes From dd10006121285b09a293d40e4f2e9de152737645 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 2 Dec 2022 13:01:28 -0500 Subject: [PATCH 224/372] Add history metadata Signed-off-by: Natalie Arellano --- text/0093-remove-shell-processes.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/text/0093-remove-shell-processes.md b/text/0093-remove-shell-processes.md index fd6d380da..ba17d59de 100644 --- a/text/0093-remove-shell-processes.md +++ b/text/0093-remove-shell-processes.md @@ -376,3 +376,28 @@ command = [""] args = [""] default = false ``` + +# History +[history]: #history + +## Amended +### Meta +[meta-1]: #meta-1 +- Name: Removed references to custom env templating +- Start Date: 2022-12-02 +- Author(s): natalieparellano +- Amendment Pull Request: (leave blank) + +### Summary + +As this is a breaking change, we decided to do this in a separate (yet to be created) RFC. + +Created issue: #258 + +In addition to the changes described originally in 0093 we'd like some way of versioning the launcher interface, to avoid surprising end-users. + +### Motivation + +Why was this amendment necessary? + +The RFC text should reflect what was actually implemented / agreed upon to avoid confusion. \ No newline at end of file From 8f41085a6837bca4e10c640572e287a6ed78114f Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Mon, 5 Dec 2022 12:32:36 -0500 Subject: [PATCH 225/372] Update status for 0083 Signed-off-by: Natalie Arellano --- text/0083-best-practices-and-guidelines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0083-best-practices-and-guidelines.md b/text/0083-best-practices-and-guidelines.md index cb87bec23..cfe9bcfe0 100644 --- a/text/0083-best-practices-and-guidelines.md +++ b/text/0083-best-practices-and-guidelines.md @@ -3,7 +3,7 @@ - Name: Best practices and guidelines for Cloud Native Buildpacks - Start Date: 2021-03-29 - Author(s): [@samj1912](https://github.com/samj1912) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#150](https://github.com/buildpacks/rfcs/pull/150) - CNB Pull Request: (leave blank) - CNB Issue: N/A From db8472c53a8f509c12d75843e901052600a233b1 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 7 Dec 2022 10:37:02 -0500 Subject: [PATCH 226/372] Update status for 0090 Signed-off-by: Natalie Arellano --- text/0090-intro-video-script.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0090-intro-video-script.md b/text/0090-intro-video-script.md index 8ebe32947..f4b2f44b0 100644 --- a/text/0090-intro-video-script.md +++ b/text/0090-intro-video-script.md @@ -3,7 +3,7 @@ - Name: Intro video script - Start Date: 2021-05-18 - Author(s): yaelharel -- Status: Approved +- Status: Implemented - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) From aaa2c6f52ef37b66aa1cf217fb7469ab4060d3a5 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Wed, 7 Dec 2022 12:20:14 -0500 Subject: [PATCH 227/372] Amend 0049 to reflect env var that was implemented Instead of `CNB_PLATFORM_DEPRECATION_MODE` and `CNB_BUILDPACK_DEPRECATION_MODE` we have just one variable, `CNB_DEPRECATION_MODE` Signed-off-by: Natalie Arellano --- text/0049-multi-api-lifecycle-descriptor.md | 36 +++++++++++++++------ 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/text/0049-multi-api-lifecycle-descriptor.md b/text/0049-multi-api-lifecycle-descriptor.md index 93763a172..d98902a4c 100644 --- a/text/0049-multi-api-lifecycle-descriptor.md +++ b/text/0049-multi-api-lifecycle-descriptor.md @@ -76,22 +76,22 @@ A builder or lifecycle image with the above descriptor file should have the foll ### Deprecated APIs Only API versions defined in a spec release can be in the deprecated range. -New `CNB_PLATFORM_DEPRECATION_MODE`, and `CNB_BUILDPACK_DEPRECATION_MODE` environment variables will control deprecation behavior with: +A new `CNB_DEPRECATION_MODE`environment variable will control deprecation behavior with: * allowed values: `warn`, `error`, `silent` * default value: `warn` **When** the `CNB_PLATFROM_API` environment variable is set to an API version in the deprecated platform API, the lifecycle shall: - - **If** `CNB_PLATFORM_DEPRECATION_MODE` is unset, **Then** print a warning and continue - - **If** `CNB_PLATFORM_DEPRECATION_MODE=warn`, **Then** print a warning and continue - - **If** `CNB_PLATFORM_DEPRECATION_MODE=error`, **Then** fail - - **If** `CNB_PLATFORM_DEPRECATION_MODE=silent`, **Then** continue w/o warning + - **If** `CNB_DEPRECATION_MODE` is unset, **Then** print a warning and continue + - **If** `CNB_DEPRECATION_MODE=warn`, **Then** print a warning and continue + - **If** `CNB_DEPRECATION_MODE=error`, **Then** fail + - **If** `CNB_DEPRECATION_MODE=silent`, **Then** continue w/o warning **When** the `api` field in a `buildpack.toml` file is set to an API version in the deprecated buildpack API range the lifecycle shall: - - **If** `CNB_BUILDPACK_DEPRECATION_MODE` is unset, **Then** print a warning and continue - - **If** `CNB_BUILDPACK_DEPRECATION_MODE=warn`, **Then** print a warning and continue - - **If** `CNB_BUILDPACK_DEPRECATION_MODE=error`, **Then** fail - - **If** `CNB_BUILDPACK_DEPRECATION_MODE=silent`, **Then** continue w/o warning + - **If** `CNB_DEPRECATION_MODE` is unset, **Then** print a warning and continue + - **If** `CNB_DEPRECATION_MODE=warn`, **Then** print a warning and continue + - **If** `CNB_DEPRECATION_MODE=error`, **Then** fail + - **If** `CNB_DEPRECATION_MODE=silent`, **Then** continue w/o warning ### Supported APIs Only API versions defined in a spec release can be in the supported range. @@ -157,3 +157,21 @@ Understanding buildpack/platform API support will require more documentation and # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes This RFC will not result in spec changes. The lifecycle descriptor file and builder labels are unspecified features. + +## Amended +### Meta +[meta-1]: #meta-1 +- Name: Variable Rename +- Start Date: 2022-12-07 +- Author(s): natalieparellano +- Amendment Pull Request: (leave blank) + +### Summary + +Instead of `CNB_PLATFORM_DEPRECATION_MODE` and `CNB_BUILDPACK_DEPRECATION_MODE` we have just one variable, `CNB_DEPRECATION_MODE`. + +### Motivation + +Why was this amendment necessary? + +Somewhere along the way, this is what we decided to implement. Updating the RFC to be accurate allows us to point end-users toward this RFC in helping to explain how APIs are deprecated. \ No newline at end of file From b8330bf54676e7f2b3a30e50235530a571b89aad Mon Sep 17 00:00:00 2001 From: Joey Brown Date: Thu, 8 Dec 2022 15:49:30 -0600 Subject: [PATCH 228/372] allow rebase by image digest Signed-off-by: Joey Brown --- text/0000-rebase-immutable-image-ref.md | 110 ++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 text/0000-rebase-immutable-image-ref.md diff --git a/text/0000-rebase-immutable-image-ref.md b/text/0000-rebase-immutable-image-ref.md new file mode 100644 index 000000000..11070a509 --- /dev/null +++ b/text/0000-rebase-immutable-image-ref.md @@ -0,0 +1,110 @@ +# Meta +[meta]: #meta +- Name: Rebase by Image Digest Reference +- Start Date: 2022-12-08 +- Author(s): [@joeybrown-sf](https://github.com/joeybrown-sf) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This is a feature for the implementation team. + +Allow passing a digest reference as rebase target. + +# Definitions +[definitions]: #definitions + +An **image reference** refers to either a **tag reference** or **digest reference**. + +A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. + +A **digest reference** refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form `/@` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. + + +# Motivation +[motivation]: #motivation + +- This feature will support a workflow where the rebase executing agent only has the digest available to it. + +# What it is +[what-it-is]: #what-it-is + +This provides a high level overview of the feature. + +- Define any new terminology. +- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor. +- Explaining the feature largely in terms of examples. +- If applicable, provide sample error messages, deprecation warnings, or migration guidance. +- If applicable, describe the differences between teaching this to existing users and new users. + +# How it Works +[how-it-works]: #how-it-works + +First, let's get some context. Today, we can execute rebase by using **tag references** but not **digest references**. + +Here are some examples of valid rebase commands using **tag references**: +1. `lifecycle rebase my-repo/foo` +1. `lifecycle rebase my-repo/foo:latest` +1. `lifecycle rebase my-repo/foo:v4` + +Here are some examples of currently invalid rebase commands using **digest references**: +1. `lifecycle rebase my-repo/foo@sha256:1234 myrepo/foo` +1. `lifecycle rebase my-repo/foo@sha256:1234 myrepo/foo:latest` +1. `lifecycle rebase my-repo/foo@sha256:1234 myrepo/foo:vNext` + +Supporting Rebase by Image Digest Reference will make both sets of commands valid. + +- When using a digest reference as the image target, there must be at least one tag reference to apply to exported image. +- **If** the `rebase target` is a `digest reference` and one or more `` inputs are provided, each `` MUST refer to the same repository as the `rebase target`. +- If a `` is included without a ``, latest will be used. + +# Migration +[migration]: #migration + +This is backwards compatible. + +# Drawbacks +[drawbacks]: #drawbacks + +# Alternatives +[alternatives]: #alternatives + +# Prior Art +[prior-art]: #prior-art + +`pack` explicitly does not support this. There is validation in `pack`: +` is not a tag reference` + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + + +# History +[history]: #history + + \ No newline at end of file From b29020c19591f4eb0de0a327498c3b6c6e62b104 Mon Sep 17 00:00:00 2001 From: Joey Brown Date: Thu, 8 Dec 2022 16:13:19 -0600 Subject: [PATCH 229/372] filled out missing section Signed-off-by: Joey Brown --- text/0000-rebase-immutable-image-ref.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/text/0000-rebase-immutable-image-ref.md b/text/0000-rebase-immutable-image-ref.md index 11070a509..3318c0bf7 100644 --- a/text/0000-rebase-immutable-image-ref.md +++ b/text/0000-rebase-immutable-image-ref.md @@ -34,18 +34,17 @@ A **digest reference** refers to a [content addressable](https://en.wikipedia.o # What it is [what-it-is]: #what-it-is -This provides a high level overview of the feature. +This is a feature to expand the lifecycle rebase command to allow targeting an image by either `tag` or `digest`. -- Define any new terminology. -- Define the target persona: buildpack author, buildpack user, platform operator, platform implementor, and/or project contributor. -- Explaining the feature largely in terms of examples. -- If applicable, provide sample error messages, deprecation warnings, or migration guidance. -- If applicable, describe the differences between teaching this to existing users and new users. +Today, we get the following error when using a digest reference: +``` +ERROR: failed to rebase: failed to write image to the following tags: [localhost:5003/foo/bar@sha256:916a9e100569ee521b86d03b8499b9b93d7d256d6e838868ae720295f2ea2f76: PUT http://localhost:5003/v2/foo/bar/manifests/sha256:916a9e100569ee521b86d03b8499b9b93d7d256d6e838868ae720295f2ea2f76: DIGEST_INVALID: provided digest did not match uploaded content] +``` # How it Works [how-it-works]: #how-it-works -First, let's get some context. Today, we can execute rebase by using **tag references** but not **digest references**. +Today, we can execute rebase by using **tag references** but not **digest references**. Here are some examples of valid rebase commands using **tag references**: 1. `lifecycle rebase my-repo/foo` @@ -60,8 +59,7 @@ Here are some examples of currently invalid rebase commands using **digest refer Supporting Rebase by Image Digest Reference will make both sets of commands valid. - When using a digest reference as the image target, there must be at least one tag reference to apply to exported image. -- **If** the `rebase target` is a `digest reference` and one or more `` inputs are provided, each `` MUST refer to the same repository as the `rebase target`. -- If a `` is included without a ``, latest will be used. +- If a `` is included without a ``, the tag `latest` will be used. # Migration [migration]: #migration From 497d904362891a57871c07e2ab86f80f1a53bb53 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 14 Dec 2022 09:09:12 -0600 Subject: [PATCH 230/372] Revert "RFC: Merge Distribution Team with Platform Team" --- text/0000-distribution-team-merge-platform.md | 104 ------------------ 1 file changed, 104 deletions(-) delete mode 100644 text/0000-distribution-team-merge-platform.md diff --git a/text/0000-distribution-team-merge-platform.md b/text/0000-distribution-team-merge-platform.md deleted file mode 100644 index 53a4917a6..000000000 --- a/text/0000-distribution-team-merge-platform.md +++ /dev/null @@ -1,104 +0,0 @@ -# Meta -[meta]: #meta -- Name: Merge Distribution Team with Platform Team -- Start Date: 2022-12-01 -- Author(s): [@jkutner](https://github.com/jkutner) -- Status: Draft -- RFC Pull Request: (leave blank) -- CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) -- Supersedes: [RFC-0062](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) - -# Summary -[summary]: #summary - -This is a proposal to consolidate two existing teams, Distribution and Platform, into a single team. - -# Definitions -[definitions]: #definitions - -* Platform - pack and any other platforms or platform components maintained by the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team) -* Distribution - tools and services that support the distribution, discovery, and integration of buildpacks. Owned by the [Distribution Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#distribution-team) - -# Motivation -[motivation]: #motivation - -The distribution team only has one maintainer, which is not sustainable and present a redunancy risk. However, the distribution team's workload is fairly small, consisting mostly of security patches and simple releases. At the same time, the Platform team has lost key resources including a maintainer and team lead. The components owned by the Distribution team align well with the Platform team's mission. For these reasons, we aim to consolidate leadership energy by merging these teams. - -# What it is -[what-it-is]: #what-it-is - -The active maintainers and contributors of the Distribution Team will move into the Platform team with the same role. The components owned by the Distribution Team will henceforth be owned by the Platform Team. A single Team Lead will continue to represent Platform Team. - -The Distribution Team will be removed from the CNB [Governance model](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md). - -# How it Works -[how-it-works]: #how-it-works - -The Platform Team will have the following members: - -* Team Lead: Terence Lee -* Maintainers: Terence Lee, Joe Kutner, David Freilich -* Contributors: All active contributors from both existing teams - -The Platform Team will own the following components: -* [pack](https://github.com/buildpacks/pack) -* [Tekton Tasks + Pipelines](https://github.com/buildpacks/tekton-integration) -* [CircleCI Pack Orb](https://github.com/buildpacks/pack-orb) -* [Buildpack Registry API](https://github.com/buildpacks/registry-api) -* [Buildpack Registry Index](https://github.com/buildpacks/registry-index) -* [Buildpack Registry Namespace Owners](https://github.com/buildpacks/registry-namespaces) -* [Github Actions](https://github.com/buildpacks/github-actions) - -# Migration -[migration]: #migration - -N/A - -# Drawbacks -[drawbacks]: #drawbacks - -* This increases the surface of the Platform Team, which already suffered from attrition. It is also one the team that owns one of the most critical CNB components: pack. This could put additional strain on its members. - -# Alternatives -[alternatives]: #alternatives - -* Do nothing - keep the same team structure. Both the Platform team and Distribution team would lack redundancy -* Disolve the Distribution Team - sunset the components it owns and find alternatives to Buildpack Registry and github-actions - -# Prior Art -[prior-art]: #prior-art - -* [Create a Distribution Team](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) -* [Announcing OpenTelemetry: the merger of OpenCensus and OpenTracing](https://cloudblogs.microsoft.com/opensource/2019/05/23/announcing-opentelemetry-cncf-merged-opencensus-opentracing/) - -# Unresolved Questions -[unresolved-questions]: #unresolved-questions - -TBD - -# Spec. Changes (OPTIONAL) -[spec-changes]: #spec-changes - -N/A - -# History -[history]: #history - - From e8323d5b9968be854c5deb25e4f52abcd4ae2a6a Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 14 Dec 2022 09:11:42 -0600 Subject: [PATCH 231/372] RFC: Merge Distribution Team with Platform Team Signed-off-by: Joe Kutner --- text/0000-distribution-team-merge-platform.md | 104 ++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 text/0000-distribution-team-merge-platform.md diff --git a/text/0000-distribution-team-merge-platform.md b/text/0000-distribution-team-merge-platform.md new file mode 100644 index 000000000..aeef2339f --- /dev/null +++ b/text/0000-distribution-team-merge-platform.md @@ -0,0 +1,104 @@ +# Meta +[meta]: #meta +- Name: Merge Distribution Team with Platform Team +- Start Date: 2022-12-01 +- Author(s): [@jkutner](https://github.com/jkutner) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: [RFC-0062](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) + +# Summary +[summary]: #summary + +This is a proposal to consolidate two existing teams, Distribution and Platform, into a single team. + +# Definitions +[definitions]: #definitions + +* Platform - pack and any other platforms or platform components maintained by the [Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#platform-team) +* Distribution - tools and services that support the distribution, discovery, and integration of buildpacks. Owned by the [Distribution Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#distribution-team) + +# Motivation +[motivation]: #motivation + +The distribution team only has one maintainer, which is not sustainable and present a redunancy risk. However, the distribution team's workload is fairly small, consisting mostly of security patches and simple releases. At the same time, the Platform team has lost key resources including a maintainer and team lead. The components owned by the Distribution team align well with the Platform team's mission. For these reasons, we aim to consolidate leadership energy by merging these teams. + +# What it is +[what-it-is]: #what-it-is + +The active maintainers and contributors of the Distribution Team will move into the Platform team with the same role. The components owned by the Distribution Team will henceforth be owned by the Platform Team. A single Team Lead will continue to represent Platform Team. + +The Distribution Team will be removed from the CNB [Governance model](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md). + +# How it Works +[how-it-works]: #how-it-works + +The Platform Team will have the following members: + +* Team Lead: Terence Lee +* Maintainers: Terence Lee, Joe Kutner, David Freilich +* Contributors: All active contributors from both existing teams + +The Platform Team will own the following components: +* [pack](https://github.com/buildpacks/pack) +* [Tekton Tasks + Pipelines](https://github.com/buildpacks/tekton-integration) +* [CircleCI Pack Orb](https://github.com/buildpacks/pack-orb) +* [Buildpack Registry API](https://github.com/buildpacks/registry-api) +* [Buildpack Registry Index](https://github.com/buildpacks/registry-index) +* [Buildpack Registry Namespace Owners](https://github.com/buildpacks/registry-namespaces) +* [Github Actions](https://github.com/buildpacks/github-actions) + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + +* This increases the surface of the Platform Team, which already suffered from attrition. It is also one the team that owns one of the most critical CNB components: pack. This could put additional strain on its members. + +# Alternatives +[alternatives]: #alternatives + +* Do nothing - keep the same team structure. Both the Platform team and Distribution team would lack redundancy +* Disolve the Distribution Team - sunset the components it owns and find alternatives to Buildpack Registry and github-actions + +# Prior Art +[prior-art]: #prior-art + +* [Create a Distribution Team](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) +* [Announcing OpenTelemetry: the merger of OpenCensus and OpenTracing](https://cloudblogs.microsoft.com/opensource/2019/05/23/announcing-opentelemetry-cncf-merged-opencensus-opentracing/) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +TBD + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +N/A + +# History +[history]: #history + + \ No newline at end of file From 1a96aca2dcb193d79b58da3facb3fcce939aca47 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 14 Dec 2022 09:20:45 -0600 Subject: [PATCH 232/372] Merge RFC 0114 Signed-off-by: Joe Kutner --- ...e-platform.md => 0114-distribution-team-merge-platform.md} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename text/{0000-distribution-team-merge-platform.md => 0114-distribution-team-merge-platform.md} (98%) diff --git a/text/0000-distribution-team-merge-platform.md b/text/0114-distribution-team-merge-platform.md similarity index 98% rename from text/0000-distribution-team-merge-platform.md rename to text/0114-distribution-team-merge-platform.md index aeef2339f..22e99e51e 100644 --- a/text/0000-distribution-team-merge-platform.md +++ b/text/0114-distribution-team-merge-platform.md @@ -4,7 +4,7 @@ - Start Date: 2022-12-01 - Author(s): [@jkutner](https://github.com/jkutner) - Status: Draft -- RFC Pull Request: (leave blank) +- RFC Pull Request: https://github.com/buildpacks/rfcs/pull/239 - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) - Supersedes: [RFC-0062](https://github.com/buildpacks/rfcs/blob/main/text/0062-distribution-team.md) @@ -101,4 +101,4 @@ A brief description of the changes. ### Motivation Why was this amendment necessary? ----> \ No newline at end of file +---> From 3964f0c4bdfb353423c898e5dbfb9a26f7fd96ed Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 15 Dec 2022 09:42:35 -0500 Subject: [PATCH 233/372] Apply suggestions from code review Co-authored-by: Sambhav Kothari Signed-off-by: Natalie Arellano --- text/0062-distribution-team.md | 2 +- text/0077-pack-buildpack-create.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0062-distribution-team.md b/text/0062-distribution-team.md index 278df08db..0404a2bfc 100644 --- a/text/0062-distribution-team.md +++ b/text/0062-distribution-team.md @@ -3,7 +3,7 @@ - Name: Create a Distribution Team - Start Date: 2020-09-03 - Author(s): [@jkutner](@jkutner) -- Status: Implemented +- Status: Superseded - RFC Pull Request: [rfcs#113](https://github.com/buildpacks/rfcs/pull/113) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/community#35](https://github.com/buildpacks/community/issues/35) diff --git a/text/0077-pack-buildpack-create.md b/text/0077-pack-buildpack-create.md index f180da574..3592796b0 100644 --- a/text/0077-pack-buildpack-create.md +++ b/text/0077-pack-buildpack-create.md @@ -3,7 +3,7 @@ - Name: Pack Command to Create a Buildpack Repo - Start Date: 2021-01-19 - Author(s): [jkutner](https://github.com/jkutner) -- Status: Implemented +- Status: Superseded - RFC Pull Request: [rfcs#136](https://github.com/buildpacks/rfcs/pull/136) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/pack#1025](https://github.com/buildpacks/pack/issues/1025) From 3200668fe8fdb7bf7f84229d2f55d94d165d3f25 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Mon, 19 Dec 2022 06:31:43 +0000 Subject: [PATCH 234/372] Fix delimiter in RFC 109 The delimiter used in an example is a colon not a double quote character. Signed-off-by: Aidan Delaney --- text/0109-build-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0109-build-config.md b/text/0109-build-config.md index d531e447e..6cf4290c4 100644 --- a/text/0109-build-config.md +++ b/text/0109-build-config.md @@ -58,7 +58,7 @@ Final value: `FOO=test` Buildpack value: `FOO=test` -Build config: `FOO.append=another-value, FOO.delim="` +Build config: `FOO.append=another-value, FOO.delim=:` Final value: `FOO=test:another-value` Buildpack value: `FOO=test` From 740522202a4c239910607f8c3524b1eb0b7c5424 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Mon, 19 Dec 2022 07:40:32 +0000 Subject: [PATCH 235/372] Rename Build Config env variable Use the name `CNB_BUILD_CONFIG_DIR` as provided in the sample implementation. Signed-off-by: Aidan Delaney --- text/0109-build-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0109-build-config.md b/text/0109-build-config.md index 6cf4290c4..0a6ebd0b2 100644 --- a/text/0109-build-config.md +++ b/text/0109-build-config.md @@ -118,5 +118,5 @@ N/A Addition of the definition of the above directory in the Platform specification i.e. - -- `CNB_CONFIG_DIR` +- `CNB_BUILD_CONFIG_DIR` - `/cnb/config/env.build` From 6f4a4a1ea6badbb83212b821ffc6703cd0b4f757 Mon Sep 17 00:00:00 2001 From: Joey Brown Date: Fri, 16 Dec 2022 09:52:56 -0600 Subject: [PATCH 236/372] pass an explicit argument of `-previous-image` Signed-off-by: Joey Brown --- text/0000-rebase-immutable-image-ref.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/text/0000-rebase-immutable-image-ref.md b/text/0000-rebase-immutable-image-ref.md index 3318c0bf7..a8874f080 100644 --- a/text/0000-rebase-immutable-image-ref.md +++ b/text/0000-rebase-immutable-image-ref.md @@ -51,15 +51,15 @@ Here are some examples of valid rebase commands using **tag references**: 1. `lifecycle rebase my-repo/foo:latest` 1. `lifecycle rebase my-repo/foo:v4` -Here are some examples of currently invalid rebase commands using **digest references**: -1. `lifecycle rebase my-repo/foo@sha256:1234 myrepo/foo` -1. `lifecycle rebase my-repo/foo@sha256:1234 myrepo/foo:latest` -1. `lifecycle rebase my-repo/foo@sha256:1234 myrepo/foo:vNext` +It is not currently possible to target an image using **digest references**. -Supporting Rebase by Image Digest Reference will make both sets of commands valid. +Supporting Rebase by Image Digest Reference will provide a mechanism to target an image rebase by tag reference or digest reference. -- When using a digest reference as the image target, there must be at least one tag reference to apply to exported image. -- If a `` is included without a ``, the tag `latest` will be used. +Here is what targeting an image via digest will look like: +1. `lifecycle rebase -previous-image my-repo/foo@sha256:1234 -tag my-repo/foo:rebase my-repo/foo` + +- When using a digest reference as the image target, there may be one or more `` to apply to exported image. If no `tag` is provided, `latest` will be used. +- If `-previous-image` is not provided, the previous is infered from the first argument, just like `analyzer`, for instance. # Migration [migration]: #migration From 1cb7ee0e397348fc66966f24d06a5657cb1e49be Mon Sep 17 00:00:00 2001 From: Joey Brown Date: Wed, 4 Jan 2023 11:47:30 -0600 Subject: [PATCH 237/372] updated some wording Signed-off-by: Joey Brown --- text/0000-rebase-immutable-image-ref.md | 26 +++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/text/0000-rebase-immutable-image-ref.md b/text/0000-rebase-immutable-image-ref.md index a8874f080..3173ba06b 100644 --- a/text/0000-rebase-immutable-image-ref.md +++ b/text/0000-rebase-immutable-image-ref.md @@ -46,20 +46,29 @@ ERROR: failed to rebase: failed to write image to the following tags: [localhost Today, we can execute rebase by using **tag references** but not **digest references**. -Here are some examples of valid rebase commands using **tag references**: -1. `lifecycle rebase my-repo/foo` -1. `lifecycle rebase my-repo/foo:latest` -1. `lifecycle rebase my-repo/foo:v4` +Here are some examples of valid rebase commands. **Tag** is `latest` if not specified: -It is not currently possible to target an image using **digest references**. +``` +lifecycle rebase my-repo/foo +``` +``` +lifecycle rebase my-repo/foo:latest +``` +``` +lifecycle rebase my-repo/foo:v4 +``` + +It is not currently possible to target an image using a **digest reference**. Supporting Rebase by Image Digest Reference will provide a mechanism to target an image rebase by tag reference or digest reference. Here is what targeting an image via digest will look like: -1. `lifecycle rebase -previous-image my-repo/foo@sha256:1234 -tag my-repo/foo:rebase my-repo/foo` +``` +lifecycle rebase -previous-image my-repo/foo@sha256:1234 -tag my-repo/foo:rebase my-repo/foo +``` - When using a digest reference as the image target, there may be one or more `` to apply to exported image. If no `tag` is provided, `latest` will be used. -- If `-previous-image` is not provided, the previous is infered from the first argument, just like `analyzer`, for instance. +- If `-previous-image` is not provided, it is infered from the first argument. This is similar behavior to `analyzer`, for instance. # Migration [migration]: #migration @@ -75,7 +84,8 @@ This is backwards compatible. # Prior Art [prior-art]: #prior-art -`pack` explicitly does not support this. There is validation in `pack`: +`pack` explicitly does not support this. There is a friendly validation message in `pack`: + ` is not a tag reference` # Unresolved Questions From 0447b0b0e5ef542678e6f117f4629aa275a568ff Mon Sep 17 00:00:00 2001 From: Joey Brown Date: Wed, 4 Jan 2023 12:01:48 -0600 Subject: [PATCH 238/372] clarified some verbiage Signed-off-by: Joey Brown --- text/0000-rebase-immutable-image-ref.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/text/0000-rebase-immutable-image-ref.md b/text/0000-rebase-immutable-image-ref.md index 3173ba06b..a21679a2c 100644 --- a/text/0000-rebase-immutable-image-ref.md +++ b/text/0000-rebase-immutable-image-ref.md @@ -12,8 +12,6 @@ # Summary [summary]: #summary -This is a feature for the implementation team. - Allow passing a digest reference as rebase target. # Definitions @@ -29,18 +27,20 @@ A **digest reference** refers to a [content addressable](https://en.wikipedia.o # Motivation [motivation]: #motivation -- This feature will support a workflow where the rebase executing agent only has the digest available to it. +Enables rebasing by targeting an immutable image digest. There are some scenarios where the **digest reference** is preferred over **tag reference**. # What it is [what-it-is]: #what-it-is This is a feature to expand the lifecycle rebase command to allow targeting an image by either `tag` or `digest`. -Today, we get the following error when using a digest reference: +Today, `lifecycle` returns the following error when appempting to use a **digest reference**: ``` ERROR: failed to rebase: failed to write image to the following tags: [localhost:5003/foo/bar@sha256:916a9e100569ee521b86d03b8499b9b93d7d256d6e838868ae720295f2ea2f76: PUT http://localhost:5003/v2/foo/bar/manifests/sha256:916a9e100569ee521b86d03b8499b9b93d7d256d6e838868ae720295f2ea2f76: DIGEST_INVALID: provided digest did not match uploaded content] ``` +This error could be avoided if digest references were permitted. + # How it Works [how-it-works]: #how-it-works @@ -60,14 +60,14 @@ lifecycle rebase my-repo/foo:v4 It is not currently possible to target an image using a **digest reference**. -Supporting Rebase by Image Digest Reference will provide a mechanism to target an image rebase by tag reference or digest reference. +_The proposed feature will provide a mechanism to target an image rebase by tag reference or digest reference._ Here is what targeting an image via digest will look like: ``` lifecycle rebase -previous-image my-repo/foo@sha256:1234 -tag my-repo/foo:rebase my-repo/foo ``` -- When using a digest reference as the image target, there may be one or more `` to apply to exported image. If no `tag` is provided, `latest` will be used. +- When using a digest reference as the image target, the caller may specify zero or more `` to apply to exported image. If no `tag` is provided, `latest` will be used. - If `-previous-image` is not provided, it is infered from the first argument. This is similar behavior to `analyzer`, for instance. # Migration From 6bf522eeee1706ef1e7b341726745fa19ddd27fb Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 5 Jan 2023 11:23:25 -0600 Subject: [PATCH 239/372] RFC 0115 [#262] Signed-off-by: Jesse Brown --- ...able-image-ref.md => 0115-rebase-immutable-image-ref.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-rebase-immutable-image-ref.md => 0115-rebase-immutable-image-ref.md} (95%) diff --git a/text/0000-rebase-immutable-image-ref.md b/text/0115-rebase-immutable-image-ref.md similarity index 95% rename from text/0000-rebase-immutable-image-ref.md rename to text/0115-rebase-immutable-image-ref.md index a21679a2c..02d548cb7 100644 --- a/text/0000-rebase-immutable-image-ref.md +++ b/text/0115-rebase-immutable-image-ref.md @@ -3,10 +3,10 @@ - Name: Rebase by Image Digest Reference - Start Date: 2022-12-08 - Author(s): [@joeybrown-sf](https://github.com/joeybrown-sf) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#262](https://github.com/buildpacks/rfcs/pull/262) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: [buildpacks/lifecycle#983](https://github.com/buildpacks/lifecycle/issues/983) - Supersedes: N/A # Summary From cd0920666082c5da5edb43e3d99dd1f3de6f16a0 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 5 Jan 2023 15:44:40 -0500 Subject: [PATCH 240/372] RFC 0116 [#216] Signed-off-by: Natalie Arellano --- ...g-cache-images.md => 0116-stop-deleting-cache-images.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-stop-deleting-cache-images.md => 0116-stop-deleting-cache-images.md} (92%) diff --git a/text/0000-stop-deleting-cache-images.md b/text/0116-stop-deleting-cache-images.md similarity index 92% rename from text/0000-stop-deleting-cache-images.md rename to text/0116-stop-deleting-cache-images.md index b90027bfe..821ca38df 100644 --- a/text/0000-stop-deleting-cache-images.md +++ b/text/0116-stop-deleting-cache-images.md @@ -3,10 +3,10 @@ - Name: Stop deleting cache images - Start Date: 2022-03-31 - Author(s): jabrown85 -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#216](https://github.com/buildpacks/rfcs/pull/216) - CNB Pull Request: (leave blank) -- CNB Issue: [buildpacks/lifecycle#803](https://github.com/buildpacks/lifecycle/issues/803) +- CNB Issue: N/A - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 9a5fb56d0558d87b9cdaf324b23b45bffc81700d Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 5 Jan 2023 16:37:32 -0500 Subject: [PATCH 241/372] Specify that creating a tracking issue is a step in the merge process Signed-off-by: Natalie Arellano --- README.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 5692aae70..919f58676 100644 --- a/README.md +++ b/README.md @@ -45,12 +45,13 @@ Once an RFC has been accepted, the sub-team maintainers should: Once an `issues-created/` label has been created for each sub-team, the RFC is ready to merge. The team member who merges the pull request should do the following: 1. Assign an id based off the pull request number. -1. Rename the file based off the ID inside `text/`. -1. Fill in the remaining metadata at the top. -1. Commit everything. -1. Update issues with RFC ID and a link to the text file. -1. Update any links in PR description to point at the committed file. -1. Remove the "Final Comment Period" label. +2. Rename the file based off the ID inside `text/`. +3. Fill in the remaining metadata at the top. +4. Commit everything. +5. Update issues with RFC ID and a link to the text file. +6. Update any links in PR description to point at the committed file. +7. Remove the "status/voting" label. +8. Create a [tracking issue](https://github.com/buildpacks/rfcs/issues/new?assignees=&labels=type%2Ftracking&template=tracking.md&title=%5BRFC+%23%3CINSERT+RFC+NUMBER+-+e.g.%2C+0099%3E%5D+-+%3CINSERT+RFC+TITLE%3E). ## Automation @@ -60,4 +61,6 @@ The `merge-rfc.sh` script automates several steps of the merge process for accep ``` Each `` should be of the form `/#` (e.g. `buildpacks/spec#1`). In the rare case that no work must be done in the project as a result of the RFC pass the `-n` flag to explicitly indicate that no issues should be linked. -After running the `merge-rfc.sh` script, manually verify the output before pushing changes. +After running the `merge-rfc.sh` script: +* Manually verify the output before pushing changes. +* Create a [tracking issue](https://github.com/buildpacks/rfcs/issues/new?assignees=&labels=type%2Ftracking&template=tracking.md&title=%5BRFC+%23%3CINSERT+RFC+NUMBER+-+e.g.%2C+0099%3E%5D+-+%3CINSERT+RFC+TITLE%3E). From b55afc0d4be7a5a1d1803090a3b3951e3a400224 Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 6 Jan 2023 09:05:06 +0000 Subject: [PATCH 242/372] Update directory in Build Config RFC We are triangulating this RFC, the implementation and the platform specification. This change makes the RFC agree with the implementation. Signed-off-by: Aidan Delaney --- text/0109-build-config.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0109-build-config.md b/text/0109-build-config.md index 0a6ebd0b2..ebb6537b5 100644 --- a/text/0109-build-config.md +++ b/text/0109-build-config.md @@ -12,7 +12,7 @@ # Summary [summary]: #summary -This RFC proposes an easy way to configure build images to allow specifying a `/cnb/config/env.build` CNB environment directory that allows updating the Buildpack `detect` and `build` environment based on the directory. +This RFC proposes an easy way to configure build images to allow specifying a `/cnb/build-config` CNB environment directory that allows updating the Buildpack `detect` and `build` environment based on the directory. # Definitions @@ -40,10 +40,10 @@ The environment variables may ideally also take precendence over any user provid # What it is [what-it-is]: #what-it-is -The RFC proposes the introduction of the following directory `/cnb/config/env.build` in build images. The directory follows the same convention as a `CNB environment directory`. The notable difference is that the environment variables sourced from this directory are applied **AFTER** processing the user-provided platform environment variables i.e. they should have the highest precedence. These variables should be available during both `detect` and `build` phases (and the `generate` phase in the future). +The RFC proposes the introduction of the following directory `/cnb/build-config/env.build` in build images. The directory follows the same convention as a `CNB environment directory`. The notable difference is that the environment variables sourced from this directory are applied **AFTER** processing the user-provided platform environment variables i.e. they should have the highest precedence. These variables should be available during both `detect` and `build` phases (and the `generate` phase in the future). -The operator can define this directory in the build image under `/cnb/config` or `CNB_CONFIG_DIR` if defined. +The operator can define this directory in the build image under `/cnb/build-config` or `CNB_BUILD_CONFIG_DIR` if defined. # How it Works [how-it-works]: #how-it-works @@ -119,4 +119,4 @@ N/A Addition of the definition of the above directory in the Platform specification i.e. - - `CNB_BUILD_CONFIG_DIR` -- `/cnb/config/env.build` +- `/cnb/build-config/` From 530ba8cda4b5a917a099180276589f46fc2c0a0d Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Fri, 6 Jan 2023 17:59:34 +0000 Subject: [PATCH 243/372] Update text/0109-build-config.md Co-authored-by: Natalie Arellano Signed-off-by: Aidan Delaney --- text/0109-build-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0109-build-config.md b/text/0109-build-config.md index ebb6537b5..fde9e5e8f 100644 --- a/text/0109-build-config.md +++ b/text/0109-build-config.md @@ -40,7 +40,7 @@ The environment variables may ideally also take precendence over any user provid # What it is [what-it-is]: #what-it-is -The RFC proposes the introduction of the following directory `/cnb/build-config/env.build` in build images. The directory follows the same convention as a `CNB environment directory`. The notable difference is that the environment variables sourced from this directory are applied **AFTER** processing the user-provided platform environment variables i.e. they should have the highest precedence. These variables should be available during both `detect` and `build` phases (and the `generate` phase in the future). +The RFC proposes the introduction of the following directory `/cnb/build-config/env` in build images. The directory follows the same convention as a `CNB environment directory`. The notable difference is that the environment variables sourced from this directory are applied **AFTER** processing the user-provided platform environment variables i.e. they should have the highest precedence. These variables should be available during both `detect` and `build` phases (and the `generate` phase in the future). The operator can define this directory in the build image under `/cnb/build-config` or `CNB_BUILD_CONFIG_DIR` if defined. From 97cb85517f7283beb92bf505696eebfa76000704 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 25 Jan 2023 17:40:39 -0600 Subject: [PATCH 244/372] initial 2023H1 Roadmap RFC Signed-off-by: Terence Lee --- text/0000-2023H1-roadmap.md | 162 ++++++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 text/0000-2023H1-roadmap.md diff --git a/text/0000-2023H1-roadmap.md b/text/0000-2023H1-roadmap.md new file mode 100644 index 000000000..0e0684e4f --- /dev/null +++ b/text/0000-2023H1-roadmap.md @@ -0,0 +1,162 @@ +# Meta +[meta]: #meta +- Name: 2023H1 Roadmap +- Start Date: 2023-01-24 +- Author(s): hone +- Status: Draft Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +This RFC details the first half of the 2023 Roadmap as well as changing the roadmaps to twice a year aligning with KubeCon EU and NA (CDD - Conference Driven Development). + +# Definitions +[definitions]: #definitions + +N/A + +# Motivation +[motivation]: #motivation + +The project has done annual roadmaps in the past, but they haven't been without their challenges. The items range from concrete to large themes. The larger scoped items like "Integration with the Cloud Native Ecosystem" is nebulous and not always clear what success look at the end of it. The roadmap has felt like a smaller vision document versus a set of prioritized items to be accomplished. + +Without clear guidance on how to shape the roadmap, it's also been a challenge to get one out (not that we're doing great this year). We missed 2022's roadmap altogether and didn't publish one until March for 2021. + +Once we did publish a roadmap, items wouldn't necessarily move forward even though we commited as a project to work on them. Items didn't always have an owner and there was no skin in the game for suggesting an idea. + +In addition, we don't do a good job of reviewing our roadmap whether that's regularly through the year or at the end of the year. + +As an incubation level project, there is an opportunity to broadcast announcements at both KubeCon EU and NA. As a project, we're often caught flatfooted with sharing highlights or announcements during these conferences. + + +# What it is +[what-it-is]: #what-it-is + +## Roadmap Changes + +For 2023, I want to propose trying something different this time around. The project will publish two roadmaps one for each half of the year aligned with work to be completed for KubeCon EU (2023-05-16) and KubeCon NA (2023-10-24). The goals I'm hoping to see: + +- Focus - Decrease the amount we're doing as a project, so we're able to deliver on the things we commit to. +- Accountability - With that focus, we also need to be accountable for what we're putting on this list. +- Marketing - Everyone involved works hard to make this project successful. We should take time to celebrate and talk about the work being done. + +### Smaller Scope + +Going forward, the roadmap will account for 6 months (in this case it's just under 4 months) worth of work. This forces items on the roadmap to be smaller and concrete since they have to be something that can be accomplished in that time frame. While it was nice having the large items, they were hard to execute and it wasn't clear what the finished state looked like. Any larger piece work will need to broken down to make it on the roadmap in a 6 month chunk. It's also not too small, that some larger chunks of work can be planned. Having a second roadmap each year, also allows us to course correct mid-year. + +### Ownership + +In order to ensure things make progress, every item in the roadmap will have an owner from the project leadership team of TOC members or team maintainers. If someone really wants something on there, they will need to volunteer themselves to help keep it on track if no one else will. This will help keep the number of items on the roadmap from ballooning. The owner will also be responsible for keeping everyone up to date. All roadmap items should link to something to GitHub where things can be tracked. + +### KubeCon Alignment + +As stated above, we'll be making roadmaps with work finishing by a KubeCon event. They're coveniently around 6 months apart give or take a few weeks. As an incubation project, we're able to share project announcements at these events. This will keep the key items we're working on top of mind, and what gets finished can easily be shared. Roadmap items can also make good talk material for the maintainer track. + +With how strict travel budgets are in the current economic climate, these events are one of the few times some of us can get together. Nothing can really replace in person conversations and brainstorming. These discussions can feed into the natural recap/review of the conference and as a way to kickstart roadmap review and planning. + +## 2023H1 Roadmap + +### Release Base Image Extension +* Owner: @natalieparellano +* Links: [RFC](https://github.com/buildpacks/rfcs/blob/main/text/0105-dockerfiles.md) + +This started out as [Stack Buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md) and now Dockerfile Extensions. Significant work has already been done on this feature over the last year. This roadmap item is about seeing this work through with releasing phase 3 in both `lifecycle` and `pack`. + +### Remove Stacks & Mixins +* Owner: @jkutner +* Links: [RFC](https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md) + +This RFC was merged in 2021 and is a dependency on Base Image Extensions. In order to get us to 1.0, we'll need to take on some of these painful backwards breaking changes in the best way possible. This work will include the Buildpack & Platform spec changes with support in `lifecycle` and `pack`. + +### Execution Environments RFC +* Owner: @hone +* Links: Coming Soon + +There has long been a desire for a "Test" support, but it's never been prioritized even though it's made the roadmap before. Not to be over ambitious, the first step is to get a RFC written and accepted. + +### Project Health +* Owner: @samj1912 +* Links: [kpack RFC](https://github.com/buildpacks/rfcs/pull/235) + +Like other [CNCF projects](https://github.com/cncf/toc/issues?q=is%3Aissue+sort%3Aupdated-desc+%22health+of%22+-label%3A%22project+onboarding%22+-label%3A%22sandbox%22+), the project has been impacted by the VMware + Broadcom acquisition. The goal of this item is to improve the general health of the project and grow contributors back to our 2020 numbers. This inludes every team having a set of active set of maintainers and contributors, thus removing the TOC needing to step in for platform. + +As for concrete items to be accomplished: + +* Establish a buildpacks-community to be used as a labs/staging area to help hype up experiments that we would be wary of investing in otherwise and if they succeed provide them authenticity in the main buildpacks org. +* Participate in mentorship programs to grow contributors like [GSoC](https://summerofcode.withgoogle.com/) and [LFX Mentorship](https://lfx.linuxfoundation.org/tools/mentorship/). + +### Pack Test Cleaning/Optimizations +* Owner: @dfreilich +* Links: [Pack Pull Request](https://github.com/buildpacks/pack/pull/1498) + +Currently, the pack acceptance tests are very complex for newcomers. In order to help with contributions, we can relax some of these tests. + +# How it Works +[how-it-works]: #how-it-works + +See [What it is](#what-it-is) for the bulk of the details. For implementing this plan: + +* Open a PR against the [community repo](https://github.com/buildpacks/community) replacing the `ROADMAP.md`. +* As part of the regular leadership meetings we will hold check ins. +* After each KubeCon there will be a recap session and kicking off the next roadmap planning. +* In 2024, we will review how this compares to the normal annual roadmap we've traditionally done. + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + +- This will be more work with twice the number of roadmap plannings. +- There will be more overhead to ensure accountability. + +# Alternatives +[alternatives]: #alternatives + +## Do Nothing + +We can continue to do the annual roadmap. This hasn't proved very successful, but we can still take the lessons learned and adjust the current process. + +# Prior Art +[prior-art]: #prior-art + +- [Buildpacks 2021 Roadmap](https://github.com/buildpacks/community/pull/72) +- [Rust 2021 Roadmap](https://blog.rust-lang.org/2020/09/03/Planning-2021-Roadmap.html) +- [TypeScript Roadmap](https://github.com/microsoft/TypeScript/wiki/Roadmap) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +N/A + +# History +[history]: #history + + From 78fecd8af0a34c5ab4024e9fccb3b5a22868edbe Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 25 Jan 2023 18:54:14 -0600 Subject: [PATCH 245/372] update scoping on roadmap RFC Signed-off-by: Terence Lee --- text/0000-2023H1-roadmap.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/0000-2023H1-roadmap.md b/text/0000-2023H1-roadmap.md index 0e0684e4f..1054db0f2 100644 --- a/text/0000-2023H1-roadmap.md +++ b/text/0000-2023H1-roadmap.md @@ -48,6 +48,10 @@ For 2023, I want to propose trying something different this time around. The pro Going forward, the roadmap will account for 6 months (in this case it's just under 4 months) worth of work. This forces items on the roadmap to be smaller and concrete since they have to be something that can be accomplished in that time frame. While it was nice having the large items, they were hard to execute and it wasn't clear what the finished state looked like. Any larger piece work will need to broken down to make it on the roadmap in a 6 month chunk. It's also not too small, that some larger chunks of work can be planned. Having a second roadmap each year, also allows us to course correct mid-year. +Not only are individual items smaller, but we as a project should commit to less so they can be accomplished aren't just a bunch of empty promises. This means hard decisions will need to be made to cut highly requested features from the roadmap. + +Since this is an OSS project, things that don't make the roadmap can still be pursued by others and welcome! With finite time of maintainers, there still may be limited support depending on the maintainer. Also, items can also bubble up during the next roadmap cycle and people should advocate for them in the GitHub discussion or slack. + ### Ownership In order to ensure things make progress, every item in the roadmap will have an owner from the project leadership team of TOC members or team maintainers. If someone really wants something on there, they will need to volunteer themselves to help keep it on track if no one else will. This will help keep the number of items on the roadmap from ballooning. The owner will also be responsible for keeping everyone up to date. All roadmap items should link to something to GitHub where things can be tracked. @@ -133,7 +137,7 @@ We can continue to do the annual roadmap. This hasn't proved very successful, bu # Unresolved Questions [unresolved-questions]: #unresolved-questions -N/A +- How many items should any one person be able to own? # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes From 49de41c21b897b07157fbbb9a5a9a4ade5a5df08 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Thu, 26 Jan 2023 18:48:04 +0000 Subject: [PATCH 246/372] Add Buildpacks community organization RFC Signed-off-by: Sambhav Kothari --- text/0000-buildpacks-community.md | 156 ++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 text/0000-buildpacks-community.md diff --git a/text/0000-buildpacks-community.md b/text/0000-buildpacks-community.md new file mode 100644 index 000000000..8d159b674 --- /dev/null +++ b/text/0000-buildpacks-community.md @@ -0,0 +1,156 @@ +# Meta +[meta]: #meta +- Name: Buildpacks Community +- Start Date: 2023-01-26 +- Author(s): [@samj1912](https://github.com/samj1912) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +The Buildpacks Community is a vendor-neutral Github organization where trusted community provided Cloud Native Buildpacks tooling, platforms and integrations can live. This would provide users a trusted place to search for Buildpack integrations maintained by the community. + + +# Definitions +[definitions]: #definitions + +- **Buildpacks Community** - The Buildpacks Community is a vendor-neutral Github organization where trusted community provided Cloud Native Buildpacks tooling, platforms and integrations can live. +- **Buildpacks Leadership** - The Buildpacks Leadership is a group of trusted individuals who are responsible for the Buildpacks Community. They have the ability to create repositories in the Buildpacks Community and approve new projects to be added to the Buildpacks Community. This will consist of the TOC and Team-leads of the Buildpacks project. +- **Buildpacks TOC** - The Buildpacks TOC is the technical oversight committee for the Buildpacks project. The TOC is responsible for the technical direction of the Buildpacks project. + + +# Motivation +[motivation]: #motivation + +There are two reasons why this community should exist. + +- The Buildpacks Community will allow for the testing of new technologies or the development of integrations in an environment that is more flexible than that of the core Buildpacks organization. This will provide a staging area for integrations that the Buildpacks team deems important but we are not yet ready to commit to long term maintanance. + +- A trusted repository of community integrations will also allow for a trusted source of integrations that solve common yet still relatively niche problems that are not suitable to be added to core Buildpacks organization. This will highlight integrations of high-quality and provide a vendor-neutral umbrella for them to live. They will also benefit from improved CI/CD resources and a common governance model. + +# What it is +[what-it-is]: #what-it-is + + + +For a project to be admitted to the Buildpacks community organization, it must meet the following criteria: + +- The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. +- The project must be open source and licensed under Apache 2.0. +- It must follow the Cloud Native Computing Foundation Code of Conduct. +- The project must enable DCO signoff for all commits. +- The project must be open to contributions and have a public issue tracker. +- The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. +- The list of project maintainers must be publicly available and controlled through a Github team. +- The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. +- All project contributors must be members of the Buildpacks community organization. +- The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). +- There should have visible automated testing for all repositories that are part of the project. +- The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. +- The project should strive have the following community health files: + - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. + - DEVELOPMENT.md: A guide to how contributors should develop the project. + - ADOPTERS.md: A list of adopters of the project. + - VERSIONING.md: A guide to how versioning is done for the project. + - RELEASE.md: A guide to how releases are done for the project. + - SECURITY.md: A guide to how security vulnerabilities should be reported. + +This criteria is meant to alleviate the following problems: + +- All projects must meet some testing standard to be trusted in order to ensure that the projects support the latest Buildpacks APIs and are actively maintained. +- All projects must have a clearly defined governance model to ensure that the project maintainers are elected from the community and that the project is open to contributions. +- There must be a defined system in place to reap abandonware. +- If a project maintainers are not making a best effort of patching out or updating vulnerable software then the project as a whole is untrustworthy. + + +# How it Works +[how-it-works]: #how-it-works + +## Project Admission + +A project can be admitted to the Buildpacks community organization by creating a Github issue in the Buildpacks community repository. The issue should contain the following information: + +- Name of the project +- Evidence for the above criteria +- A list of maintainers for the project + +The above information will be structured into an appropriate issue template. The Buildpacks Leadership will review the issue and if the project meets the above criteria, the project will be added to the Buildpacks community organization. The Buildpacks Leadership will assign a steward team to the project and the team lead of the steward team will be responsible for ensuring that the project meets the above criteria. + +Once admitted, the team lead of the steward team will create a Github team for the project and add the project maintainers to the team and mark them as the team maintainers allowing them to add other maintainers. The existing team maintainers of the steward team will be added as maintainers to the project team. + +The team lead will also create a CODEOWNERS file for the project and add the project maintainers as the code owners. + +The project maintainers will be responsible for maintaining the list of project maintainers and ensuring that all project contributors are members of the Buildpacks community organization. + +They will be able to add new Github members to the organization by creating a Github issue in the Buildpacks community [invites repository](https://github.com/buildpacks-community/invites). + +## Project Removal + +In case the project fails to meet the above criteria, the Buildpacks Leadership will work with the project maintainers to address the issues and if the project is still not ready, the project will be archived or removed from the Buildpacks community organization. + +## Project Graduation + +In case a project is deemed to be mature enough to be part of the core Buildpacks organization, the project maintainers can request for the project to be graduated to the core Buildpacks organization via the [Component Contribution RFC](https://github.com/buildpacks/rfcs/blob/main/text/0108-governance-component-maintainer-role.md). The Buildpacks TOC will review the request and if the project meets the criteria for graduation, the project will be moved to the core Buildpacks organization. + + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + + +N/A + +# Alternatives +[alternatives]: #alternatives + +N/A + +# Prior Art +[prior-art]: #prior-art + +- [CNCF Sandbox](https://www.cncf.io/sandbox-projects/) +- [Paketo Community](https://github.com/paketo-buildpacks/rfcs/blob/main/text/0008-paketo-community.md) +- [Argoproj Labs](https://github.com/argoproj-labs) +- [Crossplane Contrib](https://github.com/crossplane-contrib) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- The exact project onboarding issue template. + +# History +[history]: #history + + + From 1aa20776e8e4eb9939449d8806f309a2038bb6fe Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Fri, 27 Jan 2023 23:39:27 +0000 Subject: [PATCH 247/372] Update text/0000-buildpacks-community.md Co-authored-by: Natalie Arellano Signed-off-by: Sambhav Kothari --- text/0000-buildpacks-community.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-buildpacks-community.md b/text/0000-buildpacks-community.md index 8d159b674..44d1039cb 100644 --- a/text/0000-buildpacks-community.md +++ b/text/0000-buildpacks-community.md @@ -85,7 +85,7 @@ A project can be admitted to the Buildpacks community organization by creating a - Evidence for the above criteria - A list of maintainers for the project -The above information will be structured into an appropriate issue template. The Buildpacks Leadership will review the issue and if the project meets the above criteria, the project will be added to the Buildpacks community organization. The Buildpacks Leadership will assign a steward team to the project and the team lead of the steward team will be responsible for ensuring that the project meets the above criteria. +The above information will be structured into an appropriate issue template. The Buildpacks Leadership will review the issue and if the project meets the above criteria, the project will be added to the Buildpacks community organization. The Buildpacks Leadership will assign a team to the project and the team lead of the team will steward the project - i.e., will be responsible for ensuring that the project meets the above criteria. Once admitted, the team lead of the steward team will create a Github team for the project and add the project maintainers to the team and mark them as the team maintainers allowing them to add other maintainers. The existing team maintainers of the steward team will be added as maintainers to the project team. From c6755535c9ffe99698b4b150efed3cc3c78f235b Mon Sep 17 00:00:00 2001 From: Aidan Delaney Date: Tue, 31 Jan 2023 06:01:57 +0000 Subject: [PATCH 248/372] Update text/0000-2023H1-roadmap.md Co-authored-by: Natalie Arellano Signed-off-by: Aidan Delaney --- text/0000-2023H1-roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-2023H1-roadmap.md b/text/0000-2023H1-roadmap.md index 1054db0f2..d78d147f9 100644 --- a/text/0000-2023H1-roadmap.md +++ b/text/0000-2023H1-roadmap.md @@ -38,7 +38,7 @@ As an incubation level project, there is an opportunity to broadcast announcemen ## Roadmap Changes -For 2023, I want to propose trying something different this time around. The project will publish two roadmaps one for each half of the year aligned with work to be completed for KubeCon EU (2023-05-16) and KubeCon NA (2023-10-24). The goals I'm hoping to see: +For 2023, I want to propose trying something different this time around. The project will publish two roadmaps one for each half of the year aligned with work to be completed for KubeCon EU (2023-04-18) and KubeCon NA (2023-11-6). The goals I'm hoping to see: - Focus - Decrease the amount we're doing as a project, so we're able to deliver on the things we commit to. - Accountability - With that focus, we also need to be accountable for what we're putting on this list. From 36f5d9d25fdd8255fffefcca6e7c666818e04a99 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Tue, 31 Jan 2023 18:02:07 -0600 Subject: [PATCH 249/372] add Buildpacks Community Organization RFC Signed-off-by: Terence Lee --- text/0000-2023H1-roadmap.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-2023H1-roadmap.md b/text/0000-2023H1-roadmap.md index d78d147f9..e82d4644a 100644 --- a/text/0000-2023H1-roadmap.md +++ b/text/0000-2023H1-roadmap.md @@ -3,7 +3,7 @@ - Name: 2023H1 Roadmap - Start Date: 2023-01-24 - Author(s): hone -- Status: Draft Draft +- Status: Draft - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) @@ -84,7 +84,7 @@ There has long been a desire for a "Test" support, but it's never been prioritiz ### Project Health * Owner: @samj1912 -* Links: [kpack RFC](https://github.com/buildpacks/rfcs/pull/235) +* Links: [Buildpacks Community Organization RFC](https://github.com/buildpacks/rfcs/pull/273) Like other [CNCF projects](https://github.com/cncf/toc/issues?q=is%3Aissue+sort%3Aupdated-desc+%22health+of%22+-label%3A%22project+onboarding%22+-label%3A%22sandbox%22+), the project has been impacted by the VMware + Broadcom acquisition. The goal of this item is to improve the general health of the project and grow contributors back to our 2020 numbers. This inludes every team having a set of active set of maintainers and contributors, thus removing the TOC needing to step in for platform. From 52d7577505f71dd2f465b32068185f13fb42dea9 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 1 Feb 2023 02:17:25 -0600 Subject: [PATCH 250/372] add execution environment RFC Signed-off-by: Terence Lee --- text/0000-2023H1-roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-2023H1-roadmap.md b/text/0000-2023H1-roadmap.md index e82d4644a..317a2dd94 100644 --- a/text/0000-2023H1-roadmap.md +++ b/text/0000-2023H1-roadmap.md @@ -78,7 +78,7 @@ This RFC was merged in 2021 and is a dependency on Base Image Extensions. In ord ### Execution Environments RFC * Owner: @hone -* Links: Coming Soon +* Links: [RFC](https://github.com/buildpacks/rfcs/pull/274) There has long been a desire for a "Test" support, but it's never been prioritized even though it's made the roadmap before. Not to be over ambitious, the first step is to get a RFC written and accepted. From 51edf8dc2bafb74715faf66c32ff9255fd6b97f5 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 1 Feb 2023 07:17:28 -0800 Subject: [PATCH 251/372] Update text/0000-2023H1-roadmap.md Co-authored-by: Aidan Delaney Signed-off-by: Terence Lee --- text/0000-2023H1-roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-2023H1-roadmap.md b/text/0000-2023H1-roadmap.md index 317a2dd94..2c5538926 100644 --- a/text/0000-2023H1-roadmap.md +++ b/text/0000-2023H1-roadmap.md @@ -90,7 +90,7 @@ Like other [CNCF projects](https://github.com/cncf/toc/issues?q=is%3Aissue+sort% As for concrete items to be accomplished: -* Establish a buildpacks-community to be used as a labs/staging area to help hype up experiments that we would be wary of investing in otherwise and if they succeed provide them authenticity in the main buildpacks org. +* Establish a buildpacks-community to be used as a labs/staging area to help hype up experiments that we would be otherwise wary of investing in and, if they succeed, adopt them in the main buildpacks org. * Participate in mentorship programs to grow contributors like [GSoC](https://summerofcode.withgoogle.com/) and [LFX Mentorship](https://lfx.linuxfoundation.org/tools/mentorship/). ### Pack Test Cleaning/Optimizations From a17c023c46f24fce1686cf51244123ec3adeec41 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Wed, 1 Feb 2023 23:14:38 +0000 Subject: [PATCH 252/372] RFC 0117 [#273] Signed-off-by: Sambhav Kothari --- ...buildpacks-community.md => 0117-buildpacks-community.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-buildpacks-community.md => 0117-buildpacks-community.md} (98%) diff --git a/text/0000-buildpacks-community.md b/text/0117-buildpacks-community.md similarity index 98% rename from text/0000-buildpacks-community.md rename to text/0117-buildpacks-community.md index 44d1039cb..d92c089ab 100644 --- a/text/0000-buildpacks-community.md +++ b/text/0117-buildpacks-community.md @@ -3,10 +3,10 @@ - Name: Buildpacks Community - Start Date: 2023-01-26 - Author(s): [@samj1912](https://github.com/samj1912) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#273](https://github.com/buildpacks/rfcs/pull/273) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 733281523bc822fee8d96946fcadb2b66ae66ed3 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Fri, 3 Feb 2023 10:32:18 -0500 Subject: [PATCH 253/372] Update 0078-group-additions.md Not actually implemented as per https://cloud-native.slack.com/archives/C0331B61A1Y/p1675364009281399 Signed-off-by: Natalie Arellano --- text/0078-group-additions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0078-group-additions.md b/text/0078-group-additions.md index df01c6354..9eed25872 100644 --- a/text/0078-group-additions.md +++ b/text/0078-group-additions.md @@ -3,7 +3,7 @@ - Name: Group additions to Builder order - Start Date: 2020-12-23 - Author(s): [jkutner](@jkutner) -- Status: Implemented +- Status: Approved - RFC Pull Request: [rfcs#129](https://github.com/buildpacks/rfcs/pull/129) - CNB Pull Request: (leave blank) - CNB Issue: [buildpacks/docs#319](https://github.com/buildpacks/docs/issues/319), [buildpacks/pack#1099](https://github.com/buildpacks/pack/issues/1099), [buildpacks/pack#1100](https://github.com/buildpacks/pack/issues/1100), [buildpacks/spec#195](https://github.com/buildpacks/spec/issues/195) From b95dc2ca6818bdb7ef1f363e8db131ac3fcb8078 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 23 Feb 2022 10:22:47 -0500 Subject: [PATCH 254/372] WIP - Draft version of the RFC Signed-off-by: Juan Bustamante --- text/0000-publish-operation | 121 ++++++++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 text/0000-publish-operation diff --git a/text/0000-publish-operation b/text/0000-publish-operation new file mode 100644 index 000000000..78f14571a --- /dev/null +++ b/text/0000-publish-operation @@ -0,0 +1,121 @@ +# Meta +[meta]: #meta +- Name: Publish Operation +- Start Date: 2022-02-22 +- Author(s): Juan Bustamante (@jbustamante) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +Split the process of generating and saving it into the final destination in two different operations. +- Export: will create the image and saving it to disk in OCI layout format. When the `daemon` flag is enabled it will save all the metadata that can't be saved into the daemon in a separate report +- Publish: will take the image generated by Export and save it into the Daemon or in the Registry using the metadata report when it's necessary. + +# Definitions +[definitions]: #definitions + + +# Motivation +[motivation]: #motivation + + + +Because the [Image.Digest](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1#Image) method from the GCR library would compute the value before pushing it to a Registry, we can be sure to create the Image and add all the metadata require (annotations, cosign, etc) save it to disk and be sure that when the image is push to the registry the Image Digest will not change. (See the [thread](https://cloud-native.slack.com/archives/C033DV9EBDF/p1644523524402149) on Slack) + +Splitting the creation of the image and it's saving operation in two different process allows us the following: +- Implement an export phase with an standard output format (OCI Layout) that can be consumed by `Platforms` if they want to +- Keep consistency with the push operation behavior because the digest and the metadata can be saved linked to the Manifest +- Solve unsupported metadata cases by the Daemon, because we can save this information in a custom format together with the Image exported in OCI Layout + + +We can support the use cases: + +- OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) +- Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) + + + +# What it is +[what-it-is]: #what-it-is + +The idea is to modify the existing *Export* phase to save the image on disk in a path defined by **Platform**, in case of **Pack** the image +would be saved in a volumen shared by other phases, OCI Layout forma will be used to saved the image and a new operation called *Publish* +will be created. This operation will be invoked when the a new flag `--publish` is sent to Lifecycle and it will actually load +the image from disk generated by the export phase and then push to Daemon or Registry. + +Check the following image for a visual representation of the idea. + +```mermaid +graph LR + A[ANALIZE] --> B[RESTORE] + B --> C[BUILD] + C --> D[EXPORT] + D --> |when --publish| E[PUBLISH] + +``` + + + + + +# How it Works +[how-it-works]: #how-it-works + + + +# Migration +[migration]: #migration + + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? +- Why is this proposal the best? +- What is the impact of not doing this? + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. +Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. +This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. From ff921a6033eafe02c8f54b8eb3b9b983f1620363 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 23 Feb 2022 10:29:19 -0500 Subject: [PATCH 255/372] Fixing extension of the file Signed-off-by: Juan Bustamante --- text/{0000-publish-operation => 0000-publish-operation.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{0000-publish-operation => 0000-publish-operation.md} (100%) diff --git a/text/0000-publish-operation b/text/0000-publish-operation.md similarity index 100% rename from text/0000-publish-operation rename to text/0000-publish-operation.md From 430ad6d13495cfcd0905cfe26f4e7f77be2db478 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 24 Feb 2022 09:30:16 -0500 Subject: [PATCH 256/372] more work Signed-off-by: Juan Bustamante --- text/0000-publish-operation.md | 65 +++++++++++++++++++++++----------- 1 file changed, 44 insertions(+), 21 deletions(-) diff --git a/text/0000-publish-operation.md b/text/0000-publish-operation.md index 78f14571a..9220b08d3 100644 --- a/text/0000-publish-operation.md +++ b/text/0000-publish-operation.md @@ -12,43 +12,58 @@ # Summary [summary]: #summary -Split the process of generating and saving it into the final destination in two different operations. -- Export: will create the image and saving it to disk in OCI layout format. When the `daemon` flag is enabled it will save all the metadata that can't be saved into the daemon in a separate report -- Publish: will take the image generated by Export and save it into the Daemon or in the Registry using the metadata report when it's necessary. +Split the process of creating and saving the Image in two different operations. +- Export: will create the Image and saves it to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. When the `daemon` flag is enabled it could save all the metadata that can't be saved into the daemon in a separate report. +- Publish: will take the image generated by Export and push it into the Daemon or in the Registry. # Definitions [definitions]: #definitions - + +- A [Platform](https://buildpacks.io/docs/concepts/components/platform/) uses a lifecycle, Buildpacks (packaged in a builder), and application source code to produce an OCI image. +- A [Lifecycle](https://buildpacks.io/docs/concepts/components/lifecycle/) orchestrates Buildpacks execution, then assembles the resulting artifacts into a final app image. +- A Daemon is a service, popularized by Docker, for downloading container images, and executing and managing containers from those images. +- A Registry is a long-running service used for storing and retrieving container images. +- An [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) is the directory structure for OCI content-addressable blobs and location-addressable references. +- An [image index](https://github.com/opencontainers/image-spec/blob/main/image-index.md) is a higher-level manifest which points to a list of manifests and descriptors. +- An [Image manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md) provides a configuration and set of layers for a single container image for a specific architecture and operating system. +- A [config](https://github.com/opencontainers/image-spec/blob/main/descriptor.md) is a property references a configuration object for a container, by digest. It must support the following media type `application/vnd.oci.image.config.v1+json` # Motivation [motivation]: #motivation +Because the [Image.Digest](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1#Image) method from the GCR library would compute the Digest value before pushing it to a Registry, it allows us to create the image and add metadata require like annotations or even sign it and save it to disk and be sure that when the Image is push to the registry the Digest will not change. (See the [thread](https://cloud-native.slack.com/archives/C033DV9EBDF/p1644523524402149) on Slack). + + + +We should do this to unblock uses cases like + +- OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) +- Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) + + + + +We expect the `Export` phase to save the Image in disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format and the `Publish` operation to push it to Daemon or Registry. + + -We can support the use cases: - -- OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) -- Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) +--> - # What it is [what-it-is]: #what-it-is -The idea is to modify the existing *Export* phase to save the image on disk in a path defined by **Platform**, in case of **Pack** the image -would be saved in a volumen shared by other phases, OCI Layout forma will be used to saved the image and a new operation called *Publish* -will be created. This operation will be invoked when the a new flag `--publish` is sent to Lifecycle and it will actually load -the image from disk generated by the export phase and then push to Daemon or Registry. +An new operation called *Publish* will be added in the Lifecycle. `/cnb/lifecycle/publisher` is responsibly for pushing the Image into the Daemon or into an OCI Registry. + + +The *Export* phase will be modify to save the image on disk in a path defined by **Platform**, in case of **Pack**, for example, the image would be saved in a shared volume in the same way it's done right now, [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be used to save the Image on disk. Check the following image for a visual representation of the idea. @@ -57,12 +72,10 @@ graph LR A[ANALIZE] --> B[RESTORE] B --> C[BUILD] C --> D[EXPORT] - D --> |when --publish| E[PUBLISH] + D --> |when --push| E[PUBLISH] ``` - - # How it Works From 945b9869ef01f1f42f0f6e29a03a68dbb17da08e Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 24 Feb 2022 13:46:07 -0500 Subject: [PATCH 257/372] WIP - adding more diagrams Signed-off-by: Juan Bustamante --- text/0000-publish-operation.md | 65 +++++++++++++++++++++++++++------- 1 file changed, 53 insertions(+), 12 deletions(-) diff --git a/text/0000-publish-operation.md b/text/0000-publish-operation.md index 9220b08d3..c98ee64a1 100644 --- a/text/0000-publish-operation.md +++ b/text/0000-publish-operation.md @@ -12,9 +12,9 @@ # Summary [summary]: #summary -Split the process of creating and saving the Image in two different operations. -- Export: will create the Image and saves it to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. When the `daemon` flag is enabled it could save all the metadata that can't be saved into the daemon in a separate report. -- Publish: will take the image generated by Export and push it into the Daemon or in the Registry. +Split the process of creating and writing the Image in a final destination in two different operations. +- Export: will create the Image and saves it to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +- Publish: will take the Image generated by Export and write it into the Daemon or in the Registry. # Definitions [definitions]: #definitions @@ -60,19 +60,57 @@ We expect the `Export` phase to save the Image in disk in [OCI Image Layout](htt # What it is [what-it-is]: #what-it-is -An new operation called *Publish* will be added in the Lifecycle. `/cnb/lifecycle/publisher` is responsibly for pushing the Image into the Daemon or into an OCI Registry. +Currently the *Exporter* writes either in an OCI image registry or a docker daemon, the idea is to change that behavior to write into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. Then a new operation called *Publish* will be added in the Lifecycle. `/cnb/lifecycle/publisher` is responsibly for writing the Image either in a OCI image registry or a docker daemon. - -The *Export* phase will be modify to save the image on disk in a path defined by **Platform**, in case of **Pack**, for example, the image would be saved in a shared volume in the same way it's done right now, [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be used to save the Image on disk. - -Check the following image for a visual representation of the idea. +The new Lifecycle orchestration process will change to something like this: ```mermaid graph LR - A[ANALIZE] --> B[RESTORE] - B --> C[BUILD] - C --> D[EXPORT] - D --> |when --push| E[PUBLISH] + A[DETECT] --> B[ANALYZE] + B --> C[RESTORE] + C --> D[BUILD] + D --> E[EXPORT] + E --> |when --push| F[PUBLISH] + +``` + +This features affects the Platform implementor because they will have to include this new operation as part of the workflow to build the application image. + +Let's see some examples: + +## Examples + +### Using the Creator + +#### Pushing an image to daemon + +```=shell + /cnb/lifecycle/creator -daemon -push + +``` + +#### Pushing an image to an OCI registry + +```=shell + /cnb/lifecycle/creator -push + +``` + +### Not using the Creator + +#### Pushing an image to daemon + +```=shell + /cnb/lifecycle/exporter -output /path/to/save/the/image + /cnb/lifecycle/publisher -daemon -input /path/to/the/image [...] + +``` + +#### Pushing an image to an OCI registry + +```=shell + /cnb/lifecycle/exporter -output /path/to/save/the/image + /cnb/lifecycle/publisher -input /path/to/the/image [...] ``` @@ -100,6 +138,9 @@ When it's invoke from the Creator the current workflow is not affected because a # How it Works [how-it-works]: #how-it-works + +![](https://i.imgur.com/DLMMpGf.png) + +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +When the `Exporter` phase is invoked passing the `-daemon` flag besides writing into the Daemon also save the image to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format updating the [report.toml](https://github.com/buildpacks/spec/blob/main/platform.md#reporttoml-toml) file with all the metadata require to validate the consistency of the image when it is published to a Registry. + +# Definitions +[definitions]: #definitions + +- A [Platform](https://buildpacks.io/docs/concepts/components/platform/) uses a lifecycle, Buildpacks (packaged in a builder), and application source code to produce an OCI image. +- A [Lifecycle](https://buildpacks.io/docs/concepts/components/lifecycle/) orchestrates Buildpacks execution, then assembles the resulting artifacts into a final app image. +- A Daemon is a service, popularized by Docker, for downloading container images, and executing and managing containers from those images. +- A Registry is a long-running service used for storing and retrieving container images. +- A digest reference refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form /@ which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. +- A Image Manifest provides a configuration and set of layers for a single container image for a specific architecture and operating system. +- An [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) is the directory structure for OCI content-addressable blobs and location-addressable references. + +# Motivation +[motivation]: #motivation + +Implementing this new feature will help us to solve the problem of loosing information when the image is saved into the Daemon keeping the image on disk along with the metadata it can be used as input for other tools to offer more capabilities to the end users. + +This feature will help to unblock uses cases like +- OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) +- Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) + +# What it is +[what-it-is]: #what-it-is + +Currently the *Exporter* writes either in an OCI image registry or a docker daemon, the idea is to add the capability to write into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format only when the `-daemon` flag is used as argument AND the feature is enable using a new flag `-layout` or the default environment variable `CNB_LAYOUT_DIR`. + +## Examples + +### Exporting using the environment variable + +```=shell +> export CNB_LAYOUT_DIR=oci +> /cnb/lifecycle/exporter -daemon my-app-image +> tree /oci +. +└── oci/ + └── my-app-image/ + ├── blobs/ + │ └── sha256/ + │ └── 01.. + ├── index.json + └── oci-layout + +``` + +### Exporting using the command line flag + +```=shell +> /cnb/lifecycle/exporter -daemon -layout oci my-app-image +> tree /oci +. +└── oci/ + └── my-app-image/ + ├── blobs/ + │ └── sha256/ + │ └── 01.. + ├── index.json + └── oci-layout + +``` +As we can see there is a new folder called `oci` and inside that folder we can find our application image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format + +Attempts to use this feature when the `-daemon` flag is not used could be ignore or show some warnings messages. + +# How it Works +[how-it-works]: #how-it-works + +The lifecycle phases affected by this new behavior is: [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) + +The following new input is proposed to be added to this phase + +| Input | Environment Variable | Default Value | Description +|-------------------|-----------------------|--------------------------|---------------------- +| `` | `CNB_LAYOUT_DIR` | "" | The root directory where the OCI image will be written. The presence of a none empty value for this environment variable will enable the feature. | + + +- When the exporter is executed WITH the flag `daemon` it will check for the presence of the flag `layout` or the environment variable `CNB_LAYOUT_DIR` +- IF any of the values are present THEN + - It will create a folder name `` located at the path defined `` or `CNB_LAYOUT_DIR` + - In parallel of exporting the Image to the Daemon it will save the final Image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format in the directory created before. The image will be saved using **uncompressed** layers + - It will calculate the digest of the manifest of the compressed layers and write that value into the report.toml file + - It will update the report.toml file with all the tags and require information to verify the image once it is pushed into a registry +- OTHERWISE it will behave as it is right now + +# Migration +[migration]: #migration + + +# Drawbacks +[drawbacks]: #drawbacks + +- We could increase the disk space if we not managed the duplication of saving the layers on disk. Currently the Cache implementation (used when daemon is ON) saved the layers tarballs on disk, because the current proposal is exporting the whole image on disk it will also require more space to save the layers for the OCI format in the `blobs` folder. + +# Alternatives +[alternatives]: #alternatives + +## Redesign the current Cache + +Another potential solution could be to export the OCI image along with the current Cache implementation. The current implementation when the Daemon is enable can be describe with the following class diagram + +```mermaid +classDiagram + Cache <|-- VolumeCache + Image <|-- LocalImage + LocalImage <|-- CachingImage + CachingImage .. VolumeCache +class Cache { + <> +} + +class Image { + <> +} +``` + +When the Daemon is enabled, a `CachingImage` is created, this image implementation saves the layers tarballs in a `VolumeCache` to reuse them and increase the speed of the process. The idea could be to redesign this `Cache` implementation (VolumeCache) with a new one, maybe a `OCICache` which handles the details to avoid saving the layers tarballs duplicated. + +```mermaid +classDiagram + Cache <|-- OCICache + Image <|-- LocalImage + LocalImage <|-- CachingImage + CachingImage .. OCICache +class Cache { + <> + +} + +class Image { + <> + +} +``` +### Drawbacks + +- We will have to expose Cache implementation details to the outside world, probably spec those details, for other tools to interact with this exported data + + + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + + + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + + diff --git a/text/0000-publish-operation.md b/text/0000-publish-operation.md deleted file mode 100644 index c98ee64a1..000000000 --- a/text/0000-publish-operation.md +++ /dev/null @@ -1,185 +0,0 @@ -# Meta -[meta]: #meta -- Name: Publish Operation -- Start Date: 2022-02-22 -- Author(s): Juan Bustamante (@jbustamante) -- Status: Draft -- RFC Pull Request: (leave blank) -- CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) - -# Summary -[summary]: #summary - -Split the process of creating and writing the Image in a final destination in two different operations. -- Export: will create the Image and saves it to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -- Publish: will take the Image generated by Export and write it into the Daemon or in the Registry. - -# Definitions -[definitions]: #definitions - -- A [Platform](https://buildpacks.io/docs/concepts/components/platform/) uses a lifecycle, Buildpacks (packaged in a builder), and application source code to produce an OCI image. -- A [Lifecycle](https://buildpacks.io/docs/concepts/components/lifecycle/) orchestrates Buildpacks execution, then assembles the resulting artifacts into a final app image. -- A Daemon is a service, popularized by Docker, for downloading container images, and executing and managing containers from those images. -- A Registry is a long-running service used for storing and retrieving container images. -- An [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) is the directory structure for OCI content-addressable blobs and location-addressable references. -- An [image index](https://github.com/opencontainers/image-spec/blob/main/image-index.md) is a higher-level manifest which points to a list of manifests and descriptors. -- An [Image manifest](https://github.com/opencontainers/image-spec/blob/main/manifest.md) provides a configuration and set of layers for a single container image for a specific architecture and operating system. -- A [config](https://github.com/opencontainers/image-spec/blob/main/descriptor.md) is a property references a configuration object for a container, by digest. It must support the following media type `application/vnd.oci.image.config.v1+json` - -# Motivation -[motivation]: #motivation - -Because the [Image.Digest](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1#Image) method from the GCR library would compute the Digest value before pushing it to a Registry, it allows us to create the image and add metadata require like annotations or even sign it and save it to disk and be sure that when the Image is push to the registry the Digest will not change. (See the [thread](https://cloud-native.slack.com/archives/C033DV9EBDF/p1644523524402149) on Slack). - - - - -We should do this to unblock uses cases like - -- OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) -- Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) - - - - -We expect the `Export` phase to save the Image in disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format and the `Publish` operation to push it to Daemon or Registry. - - - - -# What it is -[what-it-is]: #what-it-is - -Currently the *Exporter* writes either in an OCI image registry or a docker daemon, the idea is to change that behavior to write into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. Then a new operation called *Publish* will be added in the Lifecycle. `/cnb/lifecycle/publisher` is responsibly for writing the Image either in a OCI image registry or a docker daemon. - -The new Lifecycle orchestration process will change to something like this: - -```mermaid -graph LR - A[DETECT] --> B[ANALYZE] - B --> C[RESTORE] - C --> D[BUILD] - D --> E[EXPORT] - E --> |when --push| F[PUBLISH] - -``` - -This features affects the Platform implementor because they will have to include this new operation as part of the workflow to build the application image. - -Let's see some examples: - -## Examples - -### Using the Creator - -#### Pushing an image to daemon - -```=shell - /cnb/lifecycle/creator -daemon -push - -``` - -#### Pushing an image to an OCI registry - -```=shell - /cnb/lifecycle/creator -push - -``` - -### Not using the Creator - -#### Pushing an image to daemon - -```=shell - /cnb/lifecycle/exporter -output /path/to/save/the/image - /cnb/lifecycle/publisher -daemon -input /path/to/the/image [...] - -``` - -#### Pushing an image to an OCI registry - -```=shell - /cnb/lifecycle/exporter -output /path/to/save/the/image - /cnb/lifecycle/publisher -input /path/to/the/image [...] - -``` - - - -# How it Works -[how-it-works]: #how-it-works - - -![](https://i.imgur.com/DLMMpGf.png) - - - -# Migration -[migration]: #migration - - -# Drawbacks -[drawbacks]: #drawbacks - -Why should we *not* do this? - -# Alternatives -[alternatives]: #alternatives - -- What other designs have been considered? -- Why is this proposal the best? -- What is the impact of not doing this? - -# Prior Art -[prior-art]: #prior-art - -Discuss prior art, both the good and bad. - -# Unresolved Questions -[unresolved-questions]: #unresolved-questions - -- What parts of the design do you expect to be resolved before this gets merged? -- What parts of the design do you expect to be resolved through implementation of the feature? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? - -# Spec. Changes (OPTIONAL) -[spec-changes]: #spec-changes -Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. -Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. -This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. From 7150fa1cd475fab3673d3308433b1aa82cda1791 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 2 Mar 2022 11:04:14 -0500 Subject: [PATCH 259/372] WIP - Adding some C4 models diagrams to complement the idea Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 8d30d551d..37095b2db 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -82,6 +82,12 @@ Attempts to use this feature when the `-daemon` flag is not used could be ignore The lifecycle phases affected by this new behavior is: [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) +At high level view of the propose solution can be summarize with the following container diagram from the C4 model + +![](https://i.imgur.com/XhtvT9j.png) + +Notice that we are relying on the OCI format Specification to expose the data for `Platforms` + The following new input is proposed to be added to this phase | Input | Environment Variable | Default Value | Description @@ -97,6 +103,8 @@ The following new input is proposed to be added to this phase - It will update the report.toml file with all the tags and require information to verify the image once it is pushed into a registry - OTHERWISE it will behave as it is right now + + # Migration [migration]: #migration @@ -113,7 +121,13 @@ This section should document breaks to public API and breaks in compatibility du ## Redesign the current Cache -Another potential solution could be to export the OCI image along with the current Cache implementation. The current implementation when the Daemon is enable can be describe with the following class diagram +Another potential solution could be to export the OCI image along with the current Cache concept and expose some contract for `Platform` to interact with this Cache and extract the final OCI image. At high level, the solution can be represented with the following container diagram from C4 model + +![](https://i.imgur.com/xl4gL1G.png) + +Notice that on this solution, because the Cache is an internal component from the Lifecycle implementation we will have to expose some kind of specification for `Platforms` to understand its format and been able to read the OCI image. + +The current implementation when the Daemon is enable can be describe with the following class diagram ```mermaid classDiagram From b56954e71887484b3c6660c1d89f68eda35d6004 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 3 Mar 2022 09:48:30 -0500 Subject: [PATCH 260/372] WIP - adding a draft idea of some changes to the report.toml file Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 37095b2db..a85869263 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -103,7 +103,28 @@ The following new input is proposed to be added to this phase - It will update the report.toml file with all the tags and require information to verify the image once it is pushed into a registry - OTHERWISE it will behave as it is right now - +#### `report.toml` (TOML) + +The new information to be added into the `report.toml` file can be summarize as follows: + +```toml +[export] +[[export.oci]] +digest = "" +manifest-size = "" +compression-algorithm = "" +library-url = "https://github.com/google/go-containerregistry/" +library-language="go" +``` +Where: +- **If** the app image was exported using the `-layer` flag, the export section will be added to the report + - `digest` MUST contain the image digest calculated based on compressed layers + - `manifest-size` MUST contain the manifest size in bytes + - `compression-algorithm` MUST contain the information of the algorithm used by GCR to compute the digest + - `library-url` MUST contain the url of the library used to export the image + - `library-language` MUST contain the client's library programming language used to export the image + +The main idea of this new section in the `report.toml` file is to provide the information of the **expected** digest of the image if it is exported to a registry using a particular implementation. This information can be used by other tools (like publish) to complete the operation and verify the consistency of the image. # Migration [migration]: #migration From 49c555cca4a8a8b4b6833a3e534c2cb5587a24f3 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 4 Mar 2022 16:29:16 -0500 Subject: [PATCH 261/372] Apply suggestions from code review Co-authored-by: Natalie Arellano Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index a85869263..83cca2bd8 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -140,13 +140,13 @@ This section should document breaks to public API and breaks in compatibility du # Alternatives [alternatives]: #alternatives -## Redesign the current Cache +## Redesign the current Launch Cache -Another potential solution could be to export the OCI image along with the current Cache concept and expose some contract for `Platform` to interact with this Cache and extract the final OCI image. At high level, the solution can be represented with the following container diagram from C4 model +Another potential solution could be to export the OCI image along with the current Launch Cache concept and expose some contract for `Platform` to interact with this Cache and extract the final OCI image. At high level, the solution can be represented with the following container diagram from C4 model ![](https://i.imgur.com/xl4gL1G.png) -Notice that on this solution, because the Cache is an internal component from the Lifecycle implementation we will have to expose some kind of specification for `Platforms` to understand its format and been able to read the OCI image. +Notice that on this solution, because the Launch Cache is an internal component from the Lifecycle implementation we will have to expose some kind of specification for `Platforms` to understand its format and been able to read the OCI image. The current implementation when the Daemon is enable can be describe with the following class diagram From 480896f6cbf09479c4b5694a70fc7905d62779ec Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 4 Mar 2022 16:55:16 -0500 Subject: [PATCH 262/372] Apply suggestions from code review Co-authored-by: Natalie Arellano Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 83cca2bd8..9c4fd1535 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -185,7 +185,7 @@ class Image { ``` ### Drawbacks -- We will have to expose Cache implementation details to the outside world, probably spec those details, for other tools to interact with this exported data +- We will have to expose Launch Cache implementation details to the outside world, probably spec those details, for other tools to interact with this exported data |Yes| B + A -->|No| END + B{IS -launch-cache defined?} -->|Yes|D + B -->|No| E + E{DOES layout-dir/image exists?} --> |Yes| L + L[...] + E --> |No| M + M[Create layout-dir/image directory] --> O[export-dir = layout-dir/image] + O --> I + D[/Warn: will export to launch cache dir/] --> F + F{DOES launch-cache/image dir exists?} -->|Yes| G + G[ ...] + F -->|No| H + H[Create launch-cache/image directory] --> N[export-dir = launch-cache/image] + N --> I[Write image to $export-dir in OCI format **] + I --> J[Calculate manifest's digest] + J --> K[/Write digest into report.toml/] + K -->END((End)) + +``` + +Notes: + - WHEN `-launch-cache` flow is executed + - The content of `blobs//` MAY contain symbolic links to content saved in the launch cache to avoid duplicating files. + - The content of `blobs//` MAY reference tar files in **uncompressed** format because that's how they are saved in the cache + - WHEN `-launch-cache` IS NOT defined + - The content of `blobs//` MAY be saved in **compressed** format + #### `report.toml` (TOML) From 6ab321df0a0ae2d64e8daf8073644ea6f7dacf6f Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 10 Mar 2022 17:43:36 -0500 Subject: [PATCH 265/372] WIP - formating issues Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index f4d92f625..6007980fc 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -166,22 +166,28 @@ The following new input is proposed to be added to this phase |-------------------|-----------------------|--------------------------|---------------------- | `` | `CNB_LAYOUT_DIR` | "" | The root directory where the OCI image will be written. The presence of a none empty value for this environment variable will enable the feature. | - -The following flowchart explains the flow proposed +Let's see the propose flow ```mermaid flowchart - A{IS -layout OR CNB_LAYOUT_DIR defined?} -->|Yes| B + A{"IS -layout OR + CNB_LAYOUT_DIR + defined?"} -->|Yes| B A -->|No| END - B{IS -launch-cache defined?} -->|Yes|D + B{"IS -launch-cache + defined?"} -->|Yes|D B -->|No| E - E{DOES layout-dir/image exists?} --> |Yes| L + E{"DOES + layout-dir/image + exist?"} --> |Yes| L L[...] E --> |No| M M[Create layout-dir/image directory] --> O[export-dir = layout-dir/image] O --> I D[/Warn: will export to launch cache dir/] --> F - F{DOES launch-cache/image dir exists?} -->|Yes| G + F{"DOES + launch-cache/image + dir exist?"} -->|Yes| G G[ ...] F -->|No| H H[Create launch-cache/image directory] --> N[export-dir = launch-cache/image] @@ -189,7 +195,6 @@ flowchart I --> J[Calculate manifest's digest] J --> K[/Write digest into report.toml/] K -->END((End)) - ``` Notes: From 710ed11b08044fe0feb74d24b3abcb34f53486e8 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 16 Mar 2022 16:46:23 -0500 Subject: [PATCH 266/372] Improvements to the flowchart - Adding feedback from review - Simplify the flowchart diagram to avoid duplicated steps - Added the pending flows in the flowchart diagram Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 52 ++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 25 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 6007980fc..6e0bd8d41 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -63,7 +63,7 @@ Or > /cnb/lifecycle/exporter -daemon -launch-cache /launch-cache -layout /oci my-app-image ``` -The expected output is the `my-app-image` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format into the `/launch-cache/my-app-image/` folder. +The expected output is the `my-app-image` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format into the `/launch-cache/oci/my-app-image/` folder. ```=shell > cd /launch-cache @@ -76,14 +76,15 @@ The expected output is the `my-app-image` exported in [OCI Image Layout](https:/ │ ├── sha256:6905011516dcf4...tar │ └── sha256:83d85471d9f8a3...tar ├── staging - └── my-app-image/ - ├── blobs/ - │ └── sha256/ - │ ├── 65d9067f915e01...tar -> /launch-cache/committed/sha256:65d9067f915e01...tar - │ ├── 6905011516dcf4...tar -> /launch-cache/committed/sha256:6905011516dcf4...tar - │ └── 83d85471d9f8a3...tar -> /launch-cache/committed/sha256:83d85471d9f8a3...tar - ├── index.json - └── oci-layout + └── oci/ + └── my-app-image/ + ├── blobs/ + │ └── sha256/ + │ ├── 65d9067f915e01...tar -> /launch-cache/committed/sha256:65d9067f915e01...tar + │ ├── 6905011516dcf4...tar -> /launch-cache/committed/sha256:6905011516dcf4...tar + │ └── 83d85471d9f8a3...tar -> /launch-cache/committed/sha256:83d85471d9f8a3...tar + ├── index.json + └── oci-layout ``` @@ -172,38 +173,39 @@ Let's see the propose flow flowchart A{"IS -layout OR CNB_LAYOUT_DIR - defined?"} -->|Yes| B + defined?"} -->|Yes| BB A -->|No| END + BB["root = "] + BB --> B B{"IS -launch-cache defined?"} -->|Yes|D - B -->|No| E - E{"DOES - layout-dir/image - exist?"} --> |Yes| L - L[...] - E --> |No| M - M[Create layout-dir/image directory] --> O[export-dir = layout-dir/image] - O --> I - D[/Warn: will export to launch cache dir/] --> F + B -->|No| GG + D[/Warn: will export to launch cache dir/] --> FF + FF["root = $launch-cache"] --> GG + GG["export-dir = $root/$layout-dir/$image"] --> F F{"DOES - launch-cache/image - dir exist?"} -->|Yes| G - G[ ...] + $export-dir + exist?"} -->|Yes| P + P["Replace image at $export-dir ‡"] + P --> J F -->|No| H - H[Create launch-cache/image directory] --> N[export-dir = launch-cache/image] - N --> I[Write image to $export-dir in OCI format **] + H[Create $export-dir directory] --> I["Write image to $export-dir †"] I --> J[Calculate manifest's digest] J --> K[/Write digest into report.toml/] K -->END((End)) ``` -Notes: +Notes **†**: - WHEN `-launch-cache` flow is executed - The content of `blobs//` MAY contain symbolic links to content saved in the launch cache to avoid duplicating files. - The content of `blobs//` MAY reference tar files in **uncompressed** format because that's how they are saved in the cache - WHEN `-launch-cache` IS NOT defined - The content of `blobs//` MAY be saved in **compressed** format +Notes **‡**: + - WHEN `the image exists in the file system` + - The idea is to use [ReplaceImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.ReplaceImage) internally this method uses [WriteImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.WriteImage) which will skip to write blobs that already exits. It means the `blobs` folder MAY contain the blobs directory MAY contain blobs which are not referenced by any of the refs, which is valid according to the OCI image specification. + - `Platforms` could include a flag to clean the directory if the user desires it #### `report.toml` (TOML) From 95dc51789144826f0cc08548a1ba07eefb9435a5 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 28 Apr 2022 16:37:27 -0500 Subject: [PATCH 267/372] Some minor fixes Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 6e0bd8d41..ab88f5a82 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -39,7 +39,7 @@ This feature will help to unblock uses cases like Currently the *Exporter* writes either in an OCI image registry or a docker daemon, the idea is to add the capability to write into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format when the new flag `-layout` or the default environment variable `CNB_LAYOUT_DIR` is set. -Let's see some examples for the propose behavior +Let's see some examples of the proposed behavior ## Examples @@ -152,9 +152,9 @@ cd oci # How it Works [how-it-works]: #how-it-works -The lifecycle phases affected by this new behavior is: [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) +The lifecycle phase affected by this new behavior is: [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) -At high level view the propose solution can be summarize with the following container diagram from the C4 model +At high level view the proposed solution can be summarize with the following container diagram from the C4 model ![](https://i.imgur.com/0OLSK8o.png) @@ -204,8 +204,8 @@ Notes **†**: Notes **‡**: - WHEN `the image exists in the file system` - - The idea is to use [ReplaceImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.ReplaceImage) internally this method uses [WriteImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.WriteImage) which will skip to write blobs that already exits. It means the `blobs` folder MAY contain the blobs directory MAY contain blobs which are not referenced by any of the refs, which is valid according to the OCI image specification. - - `Platforms` could include a flag to clean the directory if the user desires it + - The idea is to use the method [ReplaceImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.ReplaceImage). Internally this method uses [WriteImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.WriteImage) which will skip to write blobs that already exits. It means the `blobs` directory MAY contain blobs which are not referenced by any of the refs, which is valid according to the OCI image specification. + - `Platforms` could include a flag to clean the directory if the user desires it #### `report.toml` (TOML) @@ -225,13 +225,13 @@ Where: # Migration [migration]: #migration - +- No breaking changes were identified + # Drawbacks [drawbacks]: #drawbacks -- We could increase the disk space if we not managed the duplication of saving the layers on disk. Currently the Cache implementation (used when daemon is ON) saved the layers tarballs on disk, because the current proposal is exporting the whole image on disk it will also require more space to save the layers for the OCI format in the `blobs` folder. +- We could increase the disk space if we do not managed the duplication of saving the layers on disk. Currently the Cache implementation (used when daemon is ON) saved the layers tarballs on disk, the proposal is to references those layers in the image exporting on disk to avoid duplication. + # Alternatives [alternatives]: #alternatives @@ -249,6 +249,8 @@ Discuss prior art, both the good and bad. # Unresolved Questions [unresolved-questions]: #unresolved-questions +- Should be this change included on a previous RFC to handle multiple export targets int the Lifecycle? + END((End)) ``` +Notes **‡**: + - WHEN `the image exists in the file system` + - The idea is to use the method [ReplaceImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.ReplaceImage). Internally this method uses [WriteImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.WriteImage) which will skip to write blobs that already exists. It means the `blobs` directory MAY contain blobs which are not referenced by any of the refs, which is valid according to the OCI image specification. + - `Platforms` could include a flag to clean the directory if the user desires it + Notes **†**: - WHEN `-launch-cache` flow is executed - The content of `blobs//` MAY contain symbolic links to content saved in the launch cache to avoid duplicating files. @@ -202,11 +207,6 @@ Notes **†**: - WHEN `-launch-cache` IS NOT defined - The content of `blobs//` MAY be saved in **compressed** format -Notes **‡**: - - WHEN `the image exists in the file system` - - The idea is to use the method [ReplaceImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.ReplaceImage). Internally this method uses [WriteImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.WriteImage) which will skip to write blobs that already exits. It means the `blobs` directory MAY contain blobs which are not referenced by any of the refs, which is valid according to the OCI image specification. - - `Platforms` could include a flag to clean the directory if the user desires it - #### `report.toml` (TOML) The new information to be added into the `report.toml` file can be summarize as follows: From 460fae5cfd3bc1c96cf22248126ae34bf8806d14 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 6 Jun 2022 16:43:02 -0500 Subject: [PATCH 270/372] Updating the latest changes according to the latest PoC Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 240 +++++++++++++++++++------------------ 1 file changed, 122 insertions(+), 118 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index c8e09b317..712d4f845 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -1,6 +1,6 @@ # Meta [meta]: #meta -- Name: Export to OCI format when daemon is enabled +- Name: Export to OCI format - Start Date: 2022-02-22 - Author(s): Juan Bustamante (@jbustamante) - Status: Draft @@ -12,7 +12,7 @@ # Summary [summary]: #summary -When the `Exporter` phase is invoked besides writing into the Daemon or a Registry add the capability (enable explicitly by the user) to save the image to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +Add the capability to the `Exporter` phase to save the image to disk in [OCI Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. # Definitions [definitions]: #definitions @@ -28,16 +28,23 @@ When the `Exporter` phase is invoked besides writing into the Daemon or a Regist # Motivation [motivation]: #motivation -Implementing this new feature will help us to solve the problem of loosing information when the image is saved into the Daemon keeping the image on disk along with the metadata it can be used as input for other tools to offer more capabilities to the end users. - -This feature will help to unblock uses cases like -- OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) -- Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) +Exporting to [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) will enable new user's workflows in the top of this functionality. For example: + - Reduce the Lifecycle complexity removing the interaction with the Daemon. + - Solve the problem of loosing information when the image is saved into the Daemon keeping the image on disk along with the metadata generated by the Lifecycle. The OCI Image can be used as input for other tools to offer more capabilities to the end users. + - This feature will help to unblock uses cases like + - OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) + - Cosign integration. See [RFC](https://github.com/buildpacks/rfcs/pull/195) # What it is [what-it-is]: #what-it-is -Currently the *Exporter* writes either in an OCI image registry or a docker daemon, the idea is to add the capability to write into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format when the new flag `-layout` or the default environment variable `CNB_LAYOUT_DIR` is set. +The target persona affected by this change are: + +- Platform implementors: because they want to offer this feature, they will have to take care of the responsibility of: + - Pull the require dependencies (runtime image for example) and pass it through the lifecycle + - Save the resulting image to the daemon + +The general idea is to produce an [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) and save it in a file system accesible from the lifecycle execution. Let's see some examples of the proposed behavior @@ -50,162 +57,159 @@ For each case, I will present two ways of invoking the new feature: For both ways the expected output is the same -### Exporting to Daemon with launch cache enabled +### Analyzing run-image from OCI layout format + +Let's suppose the `` or `CNB_OCI_PATH` environment variable points to a directory that contains the `run-image` with the following format ```=shell -> export CNB_LAYOUT=oci -> /cnb/lifecycle/exporter -daemon -launch-cache /launch-cache my-app-image +oci +├── cnbs +   └── sample-stack-run +   ├── blobs +   │   └── sha256 +   │   ├── 1f59171fcf9a8c2a78192d4dfbf88e6f0258e24f21798ffe015c07682d3a944a +   │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e +   │   ├── 63882e64890d1adea5824dbb9bc6a812140d6b85752bf74a0a26fca78e1baf5a +   │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa +   │   └── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 +   ├── index.json +   └── oci-layout ``` -Or +And the exporter is invoked as follows ```=shell -> /cnb/lifecycle/exporter -daemon -launch-cache /launch-cache -layout /oci my-app-image +> export CNB_USE_OCI=true +> /cnb/lifecycle/analyzer -run-image cnbs/sample-stack-run:bionic my-app-image ``` -The expected output is the `my-app-image` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format into the `/launch-cache/oci/my-app-image/` folder. +Or ```=shell -> cd /launch-cache -> tree . -. -└── launch-cache/ - ├── committed/ - │ ├── io.buildpacks.lifecycle.cache.metadata - │ ├── sha256:65d9067f915e01...tar - │ ├── sha256:6905011516dcf4...tar - │ └── sha256:83d85471d9f8a3...tar - ├── staging - └── oci/ - └── my-app-image/ - ├── blobs/ - │ └── sha256/ - │ ├── 65d9067f915e01...tar -> /launch-cache/committed/sha256:65d9067f915e01...tar - │ ├── 6905011516dcf4...tar -> /launch-cache/committed/sha256:6905011516dcf4...tar - │ └── 83d85471d9f8a3...tar -> /launch-cache/committed/sha256:83d85471d9f8a3...tar - ├── index.json - └── oci-layout - +> /cnb/lifecycle/analyzer -oci -run-image cnbs/sample-stack-run:bionic my-app-image ``` -### Exporting to Daemon without launch cache enabled +The expected output is the analysis metadata [analyzed.toml](https://github.com/buildpacks/spec/blob/main/platform.md#analyzedtoml-toml) with a content similar to this one: -```=shell -> export CNB_LAYOUT=oci -> /cnb/lifecycle/exporter -daemon my-app-image +```=TOML +[run-image] + reference = "sha256:eed6d69be53111ad1d7d3f5d1c037350e6807986feb479a67f36b15f9205a56d" ``` -Or +Where the [ImageID](https://github.com/opencontainers/image-spec/blob/main/config.md#imageid) is calculated according to the OCI specification -```=shell -> /cnb/lifecycle/exporter -daemon -layout oci my-app-image -``` -The expected output is the `my-app-image` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format into the `/oci/` folder +### Exporting to OCI layout format + +Similar to the previous example, let's suppose the `` or `CNB_OCI_PATH` environment variable points to a directory that contains the `run-image` with the following format ```=shell -cd oci -> tree . -. -└── oci/ - └── my-app-image/ - ├── blobs/ - │ └── sha256/ - │ ├── 65d9067f915e01...tar - │ ├── 6905011516dcf4...tar - │ └── 83d85471d9f8a3...tar - ├── index.json - └── oci-layout +oci +├── cnbs +   └── sample-stack-run +   ├── blobs +   │   └── sha256 +   │   ├── 1f59171fcf9a8c2a78192d4dfbf88e6f0258e24f21798ffe015c07682d3a944a +   │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e +   │   ├── 63882e64890d1adea5824dbb9bc6a812140d6b85752bf74a0a26fca78e1baf5a +   │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa +   │   └── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 +   ├── index.json +   └── oci-layout ``` -### Exporting to a Registry +And the exporter is invoked as follows ```=shell > export CNB_LAYOUT=oci -> /cnb/lifecycle/exporter gcr.io/my-repo/my-app-image +> /cnb/lifecycle/exporter -run-image cnbs/sample-stack-run:bionic my-app-image ``` Or ```=shell -> /cnb/lifecycle/exporter -layout oci gcr.io/my-repo/my-app-image +> /cnb/lifecycle/exporter -oci -run-image cnbs/sample-stack-run:bionic my-app-image ``` -The expected output is the `my-app-image` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format into the `/oci/` folder +The expected output is the application `` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format ```=shell -cd oci -> tree . -. -└── oci/ - └── my-app-image/ - ├── blobs/ - │ └── sha256/ - │ ├── 65d9067f915e01...tar - │ ├── 6905011516dcf4...tar - │ └── 83d85471d9f8a3...tar - ├── index.json - └── oci-layout +├── cnbs +│   └── sample-stack-run +│   ├── blobs +│   │   └── sha256 +│   │   ├── 1f59171fcf9a8c2a78192d4dfbf88e6f0258e24f21798ffe015c07682d3a944a +│   │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e +│   │   ├── 63882e64890d1adea5824dbb9bc6a812140d6b85752bf74a0a26fca78e1baf5a +│   │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa +│   │   └── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 +│   ├── index.json +│   └── oci-layout +└── my-app-image + ├── blobs + │   └── sha256 + │   ├── 14eaea7168b1fc4b8b30f7a20f7609335cc3dbcfb6d4c1afeb1e5daefd26cdf9 + │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e -> ../../../cnbs/sample-stack-run/blobs/sha256/219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e + │   ├── 410ce2030625414163d565a56bddc6587dbc49fa2a815af5e26bb08968dec7d2 + │   ├── 54890b0245f8b234be46817f731b7b981d9ee2ea8d5a76380d91bb9abb001cdb + │   ├── 58b81a67d77e732ef07c4b995b6a7e099e5aa772d1805da7cf929d80a3fa044e + │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa -> ../../../cnbs/sample-stack-run/blobs/sha256/63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa + │   ├── 68224c8aa806b9ec8cecf2282b7b10cdf1c615785652bcc85f3a79bd06e60384 + │   ├── 6977801a3b8ac0c4a76ea22fa4dc0541fe8fc6ba964883ee0a21b5736f8acee9 + │   ├── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 -> ../../../cnbs/sample-stack-run/blobs/sha256/6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 + │   ├── 7c83d7d24ac43dbdd419d15abb849b6d155d0bc361eca0908a8ae5aefcd55557 + │   └── addd2357f3a0175410ab8f9303e747cc72af9aaeb0e7402d3fd3f144adb29db5 + ├── index.json + └── oci-layout +``` + +Also, we are proposing to use symbolic links to reference the blobs from the `run-image` this will help to save some hard drive space for the end user. + +### Using -oci flag in combination with --daemon or --publish flags + +Any combination of using multiple sources or sinks in the Lifecycle invocation of phases should throw an error to the user. For example: +```=shell +> /cnb/lifecycle/exporter -oci -daemon -run-image cnbs/sample-stack-run:bionic my-app-image + +ERROR: exporting to multiples target is not allowed ``` # How it Works [how-it-works]: #how-it-works -The lifecycle phase affected by this new behavior is: [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) +Before defining the detail behavior, lets remember some required terminology + +**Terminology** -At high level view the proposed solution can be summarize with the following container diagram from the C4 model +- An **image reference** refers to either a tag reference or digest reference. +- A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. +- A **digest reference** refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form `/@` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. -![](https://i.imgur.com/0OLSK8o.png) +The lifecycle phases affected by this new behavior are: + - [Analyze](https://buildpacks.io/docs/concepts/components/lifecycle/analyze/) + - [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) + - [Create](https://buildpacks.io/docs/concepts/components/lifecycle/create/) +At high level view the proposed solution can be summarize with the following system landscape diagram from the C4 model + +![](https://i.imgur.com/y972lTD.png) Notice that we are relying on the OCI format Specification to expose the data for `Platforms` -The following new input is proposed to be added to this phase - -| Input | Environment Variable | Default Value | Description -|-------------------|-----------------------|--------------------------|---------------------- -| `` | `CNB_LAYOUT_DIR` | "" | The root directory where the OCI image will be written. The presence of a none empty value for this environment variable will enable the feature. | - -Let's see the proposed flow - -```mermaid -flowchart - A{"IS -layout OR - CNB_LAYOUT_DIR - defined?"} -->|Yes| BB - A -->|No| END - BB["root = "] - BB --> B - B{"IS -launch-cache - defined?"} -->|Yes|D - B -->|No| GG - D[/Warn: will export to launch cache dir/] --> FF - FF["root = $launch-cache"] --> GG - GG["export-dir = $root/$layout-dir/$image"] --> F - F{"DOES - $export-dir - exist?"} -->|Yes| P - P["Replace image at $export-dir ‡"] - P --> J - F -->|No| H - H[Create $export-dir directory] --> I["Write image to $export-dir †"] - I --> J[Calculate manifest's digest] - J --> K[/Write digest into report.toml/] - K -->END((End)) -``` +The following new input are proposed to be added to these phases -Notes **‡**: - - WHEN `the image exists in the file system` - - The idea is to use the method [ReplaceImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.ReplaceImage). Internally this method uses [WriteImage](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/layout#Path.WriteImage) which will skip to write blobs that already exists. It means the `blobs` directory MAY contain blobs which are not referenced by any of the refs, which is valid according to the OCI image specification. - - `Platforms` could include a flag to clean the directory if the user desires it + | Input | Environment Variable | Default Value | Description + |-------------------|-----------------------|--------------------------|---------------------- + | `` | `CNB_USE_OCI` | false | Analyze or Export image from/to OCI layout format on disk | + | `` | `CNB_OCI_PATH` | /oci | Path to oci directory where the images are saved | -Notes **†**: - - WHEN `-launch-cache` flow is executed - - The content of `blobs//` MAY contain symbolic links to content saved in the launch cache to avoid duplicating files. - - The content of `blobs//` MAY reference tar files in **uncompressed** format because that's how they are saved in the cache - - WHEN `-launch-cache` IS NOT defined - - The content of `blobs//` MAY be saved in **compressed** format +- WHEN `the new flag -oci or the default environment variable CNB_USE_OCI are set to true` during the phase invocation THEN the feature will be enabled. +The image look up will be done following this rules: + - WHEN `the image points to a tag reference` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - WHEN `the image points to a digest reference` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` #### `report.toml` (TOML) @@ -218,7 +222,7 @@ digest = "" manifest-size = "" ``` Where: -- **If** the app image was exported using the `-layer` flag, the export section will be added to the report +- **If** the app image was exported to OCI format on disk, the export section will be added to the report - `digest` MUST contain the image digest calculated based on compressed layers - `manifest-size` MUST contain the manifest size in bytes From 580c8ba85b5ab67a9bf5bade7198c16f945fc8cc Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 6 Jun 2022 16:54:56 -0500 Subject: [PATCH 271/372] Formatting changes Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 712d4f845..1f4bc4d9c 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -19,10 +19,12 @@ Add the capability to the `Exporter` phase to save the image to disk in [OCI Lay - A [Platform](https://buildpacks.io/docs/concepts/components/platform/) uses a lifecycle, Buildpacks (packaged in a builder), and application source code to produce an OCI image. - A [Lifecycle](https://buildpacks.io/docs/concepts/components/lifecycle/) orchestrates Buildpacks execution, then assembles the resulting artifacts into a final app image. -- A Daemon is a service, popularized by Docker, for downloading container images, and executing and managing containers from those images. -- A Registry is a long-running service used for storing and retrieving container images. -- A digest reference refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form /@ which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. -- A Image Manifest provides a configuration and set of layers for a single container image for a specific architecture and operating system. +- A **Daemon** is a service, popularized by Docker, for downloading container images, and executing and managing containers from those images. +- A **Registry** is a long-running service used for storing and retrieving container images. +- An **image reference** refers to either a tag reference or digest reference. +- A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. +- A **digest reference** refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form `/@` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. +- A **image Manifest** provides a configuration and set of layers for a single container image for a specific architecture and operating system. - An [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) is the directory structure for OCI content-addressable blobs and location-addressable references. # Motivation @@ -40,11 +42,10 @@ Exporting to [OCI Image Layout](https://github.com/opencontainers/image-spec/blo The target persona affected by this change are: -- Platform implementors: because they want to offer this feature, they will have to take care of the responsibility of: +- **Platform implementors**: they will have to take care of the responsibility of: - Pull the require dependencies (runtime image for example) and pass it through the lifecycle - - Save the resulting image to the daemon -The general idea is to produce an [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) and save it in a file system accesible from the lifecycle execution. +The general idea is to produce an [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) and save it in a file system accesible from the lifecycle execution. Let's see some examples of the proposed behavior @@ -178,14 +179,6 @@ ERROR: exporting to multiples target is not allowed # How it Works [how-it-works]: #how-it-works -Before defining the detail behavior, lets remember some required terminology - -**Terminology** - -- An **image reference** refers to either a tag reference or digest reference. -- A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. -- A **digest reference** refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form `/@` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. - The lifecycle phases affected by this new behavior are: - [Analyze](https://buildpacks.io/docs/concepts/components/lifecycle/analyze/) - [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) From d7f4ec80bd6c937b5eb4db0faa6a56a4944f1d1c Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 6 Jun 2022 17:12:44 -0500 Subject: [PATCH 272/372] Some other changes Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 1f4bc4d9c..28b59c83c 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -45,7 +45,7 @@ The target persona affected by this change are: - **Platform implementors**: they will have to take care of the responsibility of: - Pull the require dependencies (runtime image for example) and pass it through the lifecycle -The general idea is to produce an [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) and save it in a file system accesible from the lifecycle execution. +The general idea is to produce an image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format and save it in a file system accesible from the lifecycle execution. Let's see some examples of the proposed behavior @@ -77,7 +77,7 @@ oci    └── oci-layout ``` -And the exporter is invoked as follows +And the analyzer is invoked as follows ```=shell > export CNB_USE_OCI=true @@ -203,10 +203,11 @@ The image look up will be done following this rules: - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - WHEN `the image points to a digest reference` - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - A part from the look up, the logic for each phase should remain the same #### `report.toml` (TOML) -The new information to be added into the `report.toml` file can be summarize as follows: +The new information to be added into the `report.toml` file can be summarize as follows: ```toml [export] @@ -227,27 +228,24 @@ Where: # Drawbacks [drawbacks]: #drawbacks -- We could increase the disk space if we do not managed the duplication of saving the layers on disk. Currently the Cache implementation (used when daemon is ON) saved the layers tarballs on disk, the proposal is to references those layers in the image exporting on disk to avoid duplication. - +- We could increase the disk space if we do not managed the duplication of saving the layers on disk. The proposal suggest to use symbolic links to reference layers on disk and avoid duplication. # Alternatives [alternatives]: #alternatives - - +- What other designs have been considered? + - Doing nothing, just keep exporting only to Daemon or Registry +- Why is this proposal the best? [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format is an standard from which other tools can create a [OCI Runtime Specification bundle](https://github.com/opencontainers/runtime-spec/blob/v1.0.0/bundle.md) exporting to this format enables Platforms to implement any feature in the top of this format +- What is the impact of not doing this? Probably will never remove the Daemon support in the Lifecycle # Prior Art [prior-art]: #prior-art -Discuss prior art, both the good and bad. +- Discussion around removing the Daemon support [RFC](https://github.com/buildpacks/rfcs/blob/jjbustamante/feature/deprecate-daemon/text/0000-deprecate-daemon.md) # Unresolved Questions [unresolved-questions]: #unresolved-questions -- Should this change be included on a previous RFC to handle multiple export targets int the Lifecycle? -- Currently the *Launch Cache* saves uncompressed tarballs. Is this by design? Is there any reason for those tarballs to do not be saved compressed? -Let's suppose the `` or `CNB_OCI_PATH` environment variable points to a directory that contains the `run-image` with the following format +Let's suppose a directory exits and it has the following structure: ```=shell oci -├── cnbs -   └── sample-stack-run -   ├── blobs -   │   └── sha256 -   │   ├── 1f59171fcf9a8c2a78192d4dfbf88e6f0258e24f21798ffe015c07682d3a944a -   │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e -   │   ├── 63882e64890d1adea5824dbb9bc6a812140d6b85752bf74a0a26fca78e1baf5a -   │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa -   │   └── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 -   ├── index.json -   └── oci-layout +├── cnb +│ └── my-stack-run +│ ├── blobs +│ │ └── sha256 +│ │ └── 1bcd..x +│ ├── index.json +│ └── oci-layout +├── foo +│ └── my-cache +│ ├── blobs +│ │ └── sha256 +│ │ └── 1f591..a +│ ├── index.json +│ └── oci-layout +└── bar + └── my-previous-app + ├── blobs + │ └── sha256 + │ └── 1efg..w + ├── index.json + └── oci-layout ``` -And the analyzer is invoked as follows +For each case, I will present two ways of enabling the new capability: + +- Using an environment Variable +- Using a new `oci` flag + +In any case the expected output is the same. + +#### Analyze phase + +##### Analyzing run-image ```=shell > export CNB_USE_OCI=true -> /cnb/lifecycle/analyzer -run-image cnbs/sample-stack-run:bionic my-app-image -``` +> /cnb/lifecycle/analyzer -run-image cnb/my-stack-run:bionic my-app-image -Or +# OR -```=shell -> /cnb/lifecycle/analyzer -oci -run-image cnbs/sample-stack-run:bionic my-app-image -``` +> /cnb/lifecycle/analyzer -oci -run-image cnb/my-stack-run:bionic my-app-image -The expected output is the analysis metadata [analyzed.toml](https://github.com/buildpacks/spec/blob/main/platform.md#analyzedtoml-toml) with a content similar to this one: +# expected output +# analyzed.toml file with the usual `run-image.reference` -```=TOML -[run-image] - reference = "sha256:eed6d69be53111ad1d7d3f5d1c037350e6807986feb479a67f36b15f9205a56d" ``` -Where the [ImageID](https://github.com/opencontainers/image-spec/blob/main/config.md#imageid) is calculated according to the OCI specification +##### Analyzing previous-image + ```=shell +> export CNB_USE_OCI=true +> /cnb/lifecycle/analyzer -run-image cnb/my-stack-run:bionic -previous-image bar/my-previous-app my-app-image + +# OR -### Exporting to OCI layout format +> /cnb/lifecycle/analyzer -oci -run-image cnb/my-stack-run:bionic-previous-image bar/my-previous-app my-app-image -Similar to the previous example, let's suppose the `` or `CNB_OCI_PATH` environment variable points to a directory that contains the `run-image` with the following format +# expected output +# analyzed.toml file with the usual `run-image.reference` and `previos-image.reference` +``` + + +##### Analyzing run-image not saved on disk ```=shell > export CNB_USE_OCI=true -> /cnb/lifecycle/exporter -run-image cnbs/sample-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -run-image cnb/bad-run-image my-app-image + +# OR + +> /cnb/lifecycle/analyzer -oci -run-image cnb/bad-run-image my-app-image + +# expected output + +ERROR: the run-image could not be found at path: /oci/cnb/bad-run-image ``` -Or +##### Analyzing without run-image argument ```=shell -> /cnb/lifecycle/exporter -oci -run-image cnbs/sample-stack-run:bionic my-app-image +> export CNB_USE_OCI=true +> /cnb/lifecycle/analyzer my-app-image + +# OR + +> /cnb/lifecycle/analyzer -oci my-app-image + +# expected output + +ERROR: -run-image is required when OCI feature is enabled ``` -The expected output is the application `` exported in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format +#### Export phase + +##### Export to OCI + ```=shell +> export CNB_USE_OCI=true +> /cnb/lifecycle/exporter -run-image cnb/my-stack-run:bionic my-app-image + +# OR + +> /cnb/lifecycle/exporter -oci -run-image cnb/my-stack-run:bionic my-app-image + +# expected output +# the store folder will be updated with the application image as follows + oci -├── cnbs - │   └── sample-stack-run - │   ├── blobs - │   │   └── sha256 - │   │   ├── 1f59171fcf9a8c2a78192d4dfbf88e6f0258e24f21798ffe015c07682d3a944a - │   │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e - │   │   ├── 63882e64890d1adea5824dbb9bc6a812140d6b85752bf74a0a26fca78e1baf5a - │   │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa - │   │   └── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 - │   ├── index.json - │   └── oci-layout - └── my-app-image - ├── blobs - │   └── sha256 - │   ├── 14eaea7168b1fc4b8b30f7a20f7609335cc3dbcfb6d4c1afeb1e5daefd26cdf9 - │   ├── 219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e -> ../../../cnbs/sample-stack-run/blobs/sha256/219480c73c20efd82d425207e7555eba09cb113c845250cef09ba376e1dd506e - │   ├── 410ce2030625414163d565a56bddc6587dbc49fa2a815af5e26bb08968dec7d2 - │   ├── 54890b0245f8b234be46817f731b7b981d9ee2ea8d5a76380d91bb9abb001cdb - │   ├── 58b81a67d77e732ef07c4b995b6a7e099e5aa772d1805da7cf929d80a3fa044e - │   ├── 63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa -> ../../../cnbs/sample-stack-run/blobs/sha256/63cb781c9d22ed765584437fea6db568f79be248ce40fc6695017d3ef3e8caaa - │   ├── 68224c8aa806b9ec8cecf2282b7b10cdf1c615785652bcc85f3a79bd06e60384 - │   ├── 6977801a3b8ac0c4a76ea22fa4dc0541fe8fc6ba964883ee0a21b5736f8acee9 - │   ├── 6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 -> ../../../cnbs/sample-stack-run/blobs/sha256/6b534265b45a4c09a5e05dde5ccf2450ea505481658a1c9ae9e0aae32362e941 - │   ├── 7c83d7d24ac43dbdd419d15abb849b6d155d0bc361eca0908a8ae5aefcd55557 - │   └── addd2357f3a0175410ab8f9303e747cc72af9aaeb0e7402d3fd3f144adb29db5 - ├── index.json - └── oci-layout -``` +├── cnb +│ └── my-stack-run +│ ├── blobs +│ │ └── sha256 +│ │ └── 1bcd..x +│ ├── index.json +│ └── oci-layout +├── foo +│ └── // omiting for simplicity +├── bar +│ └── // omiting for simplicity +└── my-app-image + └── blobs + ├── sha256 + │ ├── 1bcd..x + │ ├── 2f789..d + │ └── 3g234..f + ├── index.json + └── oci-layout -Also, we are proposing to use symbolic links to reference the blobs from the `run-image` as this will help to save some hard drive space for the end user. +``` -### Using -oci flag in combination with --daemon or --publish flags +##### Using -oci flag in combination with --daemon or --publish flags Any combination of using multiple sources or sinks in the Lifecycle invocation of phases should throw an error to the user. For example: ```=shell -> /cnb/lifecycle/exporter -oci -daemon -run-image cnbs/sample-stack-run:bionic my-app-image +> export CNB_USE_OCI=true +> /cnb/lifecycle/exporter -daemon -run-image cnb/my-stack-run:bionic my-app-image + +# OR + +> /cnb/lifecycle/exporter -oci -daemon -run-image cnb/my-stack-run:bionic my-app-image ERROR: exporting to multiples target is not allowed ``` @@ -197,8 +264,8 @@ Notice that we are relying on the OCI format Specification to expose the data fo The following new inputs are proposed to be added to these phases - | Input | Environment Variable | Default Value | Description - |-------------------|-----------------------|--------------------------|---------------------- + | Input | Environment Variable | Default Value | Description + |-------|-----------------------|---------------|-------------- | `` | `CNB_USE_OCI` | false | Analyze or Export image from/to OCI layout format on disk | | `` | `CNB_OCI_PATH` | /oci | Path to oci directory where the images are saved | From f9e225f8108b8496a6a77d81fd079a73ef3ab7e7 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 31 Oct 2022 12:09:03 -0500 Subject: [PATCH 298/372] removing the plan for daemon removal Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 22 ++++------------------ 1 file changed, 4 insertions(+), 18 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index a4ac5cc82..6fb8a9346 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -2,7 +2,7 @@ [meta]: #meta - Name: Export to OCI format - Start Date: 2022-02-22 -- Author(s): Juan Bustamante (@jbustamante) +- Author(s): Juan Bustamante (@jjbustamante) - Status: Draft - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) @@ -277,6 +277,9 @@ The image look up will be done following these rules: - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - Apart from the look up, the logic for each phase should remain the same + + + ## Proof of concept In order to validate the feasibility of the proposed feature, we developed a proof of concept with one of the most important side effects this capability can add into the project: **Removing the Daemon Support**. You can also check a recording with the demo in the following [link](https://drive.google.com/file/d/1W1125OHuyUlx88BRroUTLBfrFHhFM5A9/view?usp=sharing) @@ -334,23 +337,6 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is - No breaking changes were identified -## For removing Daemon support in near future - -I propose the following high level strategy to accomplish the goal - -- Approve this RFC and add the export to OCI layout format feature in Lifecycle -- Implement in Pack a new workflow when the `-daemon` flag is enabled based on what we did with the PoC. The workflow could be enabled by the user in **experimental mode** - - Make noise in the community so people can help us using it - - Collect feedback about it - - Correct issues until we feel confortable -- Turn off the actual `-daemon` implementation and replace it with the `-oci` by default in Pack - - Maybe we can also add a new flag to **re-use the old implementation** in case the user needs it - - Keep doing noise in the community - - Collect feedback about it - - Correct issues until we feel confortable -- Schedule the removal of the Daemon support in Lifecycle version X.Y.Z. -- Remove the actual daemon support from the code base for Lifecycle version X.Y.Z - # Drawbacks [drawbacks]: #drawbacks From f79bb237770c38fbaeec5a6915ffa13787c6b691 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 3 Nov 2022 09:46:41 -0500 Subject: [PATCH 299/372] adding the possibility to allow partial images on disk Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 6fb8a9346..7c922ac74 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -43,7 +43,7 @@ The current process, executed by the lifecycle, does not take into consideration ### What use cases does it support? Exporting to [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) will enable new user's workflows on top of this functionality. For example: - - Reduce the Lifecycle complexity removing the interaction with the Daemon. + - It provides a mechanism to reduce the Lifecycle complexity by removing the interaction with the Daemon in the future. - Solve the problem of losing information when the image is saved into the Daemon, keeping the image on disk along with the metadata generated by the Lifecycle. The OCI Image can be used as input for other tools to offer more capabilities to the end users. - This feature will help to unblock uses cases like - OCI annotations. See [RFC](https://github.com/buildpacks/rfcs/pull/196) @@ -52,7 +52,11 @@ Exporting to [OCI Image Layout](https://github.com/opencontainers/image-spec/blo ### What is the expected outcome? -Lifecycle will be capable of exporting application image into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +Lifecycle will be capable of exporting the application image into disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. The image saved on disk could have the following considerations: + +- The `blobs` directory MAY be missing `base image` or `run image` blobs. These layers may not be needed on disk as they could be already accessible in a blob store. +- The `blobs` directory SHOULD always have buildpacks generated `blobs`. + # What it is [what-it-is]: #what-it-is @@ -63,6 +67,8 @@ The target personas affected by this change are: - **Platform implementors**: they will have to take care of the responsibility of creating a store resource on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format and pass it through the lifecycle during the phases execution. +The process of writing any image on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be expensive in terms of hard drive space or IO operation (compressing or uncompressing layers). In order to provide flexibility for the implementation, the `analyzer` or `exporter` binaries only require the *Image Manifest* and the *Image Config* to execute their operations, based on this, we proposed the Lifecycle can be configured to work with a partial representation of the images on disk, meaning that some blobs MAY be missing (which is ok according to the [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format). The missing blobs COULD be those that are already available in a daemon or registry. + Let's see some examples of the proposed behavior ## Examples From 966488a11a5690c8112f6794ac2a8c6caa34a556 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 3 Nov 2022 18:19:29 -0500 Subject: [PATCH 300/372] extending the examples section Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 256 +++++++++++++++++++++++++++++++------ 1 file changed, 215 insertions(+), 41 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 7c922ac74..be61b28dc 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -67,7 +67,7 @@ The target personas affected by this change are: - **Platform implementors**: they will have to take care of the responsibility of creating a store resource on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format and pass it through the lifecycle during the phases execution. -The process of writing any image on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be expensive in terms of hard drive space or IO operation (compressing or uncompressing layers). In order to provide flexibility for the implementation, the `analyzer` or `exporter` binaries only require the *Image Manifest* and the *Image Config* to execute their operations, based on this, we proposed the Lifecycle can be configured to work with a partial representation of the images on disk, meaning that some blobs MAY be missing (which is ok according to the [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format). The missing blobs COULD be those that are already available in a daemon or registry. +The process of writing any image on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be expensive in terms of hard drive space or IO operation (compressing or uncompressing layers). In order to provide flexibility for the implementation, the `analyzer` or `exporter` binaries only require the *Image Manifest* and the *Image Config* to execute their operations, based on this, we proposed the Lifecycle can be configured to work with a partial representation of the images on disk, meaning that some blobs MAY be missing (which is ok according to the [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format). The missing blobs SHOULD be those that are already available in a daemon or registry. Let's see some examples of the proposed behavior @@ -92,10 +92,21 @@ Let's suppose a directory exits and it has the following structure: ```=shell oci ├── cnb -│ └── my-stack-run +│ ├── my-full-stack-run +│ │ ├── blobs +│ │ │ └── sha256 +│ │ │ ├── 1f59...944a // manifest +│ │ │ ├── 6388...af5a // config +│ │ │ ├── 824b...f984e +│ │ │ ├── f5f7...5b38 +│ │ │ └── 870e...f1b09 +│ │ ├── index.json +│ │ └── oci-layout +│ └── my-partial-stack-run │ ├── blobs │ │ └── sha256 -│ │ └── 1bcd..x +│ │ ├── 1f59...944a // manifest +│ │ └── 6388...af5a // config │ ├── index.json │ └── oci-layout ├── foo @@ -105,16 +116,29 @@ oci │ │ └── 1f591..a │ ├── index.json │ └── oci-layout -└── bar - └── my-previous-app - ├── blobs - │ └── sha256 - │ └── 1efg..w +├── bar +│ └── my-previous-app +│ ├── blobs +│ │ └── sha256 +│ │ └── 1efg..w +│ ├── index.json +│ └── oci-layout +└── my-app-image + └── blobs + ├── sha256 + │ ├── 1bcd5..x // app image manifest + │ ├── 2f789..d // app image config + │ ├── 824b...f984e // run layer + │ ├── f5f7...5b38 // run layer + │ ├── 870e...f1b09 // run layer + │ └── 3g234..f // buildpack layer ├── index.json └── oci-layout ``` -For each case, I will present two ways of enabling the new capability: +The images named **cnb/my-full-stack-run** and **cnb/my-partial-stack-run** represents the same image but the partial one has missing `blobs`, those `blobs` are the layers that are already available in the store from it came from. + +For each example case, I will present two ways of enabling the new capability: - Using an environment Variable - Using a new `oci` flag @@ -123,52 +147,51 @@ In any case the expected output is the same. #### Analyze phase -##### Analyzing run-image +##### Analyzing run-image full saved on disk ```=shell > export CNB_USE_OCI=true -> /cnb/lifecycle/analyzer -run-image cnb/my-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -oci -run-image cnb/my-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -oci -run-image cnb/my-full-stack-run:bionic my-app-image # expected output # analyzed.toml file with the usual `run-image.reference` ``` -##### Analyzing previous-image +##### Analyzing run-image partial saved on disk - ```=shell +```=shell > export CNB_USE_OCI=true -> /cnb/lifecycle/analyzer -run-image cnb/my-stack-run:bionic -previous-image bar/my-previous-app my-app-image +> /cnb/lifecycle/analyzer -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -oci -run-image cnb/my-stack-run:bionic-previous-image bar/my-previous-app my-app-image +> /cnb/lifecycle/analyzer -oci -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image # expected output -# analyzed.toml file with the usual `run-image.reference` and `previos-image.reference` +# analyzed.toml file with the usual `run-image.reference` +# there should be no change behavior in the phase even thought the image has missing blobs + ``` - - ##### Analyzing run-image not saved on disk ```=shell @@ -201,54 +224,96 @@ ERROR: -run-image is required when OCI feature is enabled #### Export phase -##### Export to OCI +##### Export to OCI using run-image full saved on disk +```=shell +> export CNB_USE_OCI=true +> /cnb/lifecycle/exporter -run-image cnb/my-full-stack-run:bionic my-app-image + +# OR + +> /cnb/lifecycle/exporter -oci -run-image cnb/my-full-stack-run:bionic my-app-image + +# expected output +# the store folder will be updated with the application image as follows + +oci +├── cnb +│ └── my-full-stack-run +│ ├── blobs +│ │ └── sha256 +│ │ ├── 1f59...944a // manifest +│ │ ├── 6388...af5a // config +│ │ ├── 824b...f984e +│ │ ├── f5f7...5b38 +│ │ └── 870e...f1b09 +│ ├── index.json +│ └── oci-layout +├── // some folders omitted for simplicity +└── my-app-image + └── blobs + ├── sha256 + │ ├── 1bcd5..x // app image manifest + │ ├── 2f789..d // app image config + │ ├── 824b...f984e // run layer + │ ├── f5f7...5b38 // run layer + │ ├── 870e...f1b09 // run layer + │ └── 3g234..f // buildpack layer + ├── index.json + └── oci-layout + +``` + +As we can see, the application image `my-app-image` contains a **full** copy of the layers in its `blobs` folder. + + +##### Export to OCI using run-image partially saved on disk ```=shell > export CNB_USE_OCI=true -> /cnb/lifecycle/exporter -run-image cnb/my-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -run-image cnb/my-partial-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/exporter -oci -run-image cnb/my-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -oci -run-image cnb/my-partial-stack-run:bionic my-app-image # expected output # the store folder will be updated with the application image as follows oci ├── cnb -│ └── my-stack-run +│ └── my-partial-stack-run │ ├── blobs │ │ └── sha256 -│ │ └── 1bcd..x +│ │ ├── 1f59...944a // manifest +│ │ ├── 6388...af5a // config │ ├── index.json │ └── oci-layout -├── foo -│ └── // omiting for simplicity -├── bar -│ └── // omiting for simplicity +├── // some folders omitted for simplicity └── my-app-image └── blobs ├── sha256 - │ ├── 1bcd..x - │ ├── 2f789..d - │ └── 3g234..f + │ ├── 1bcd5..x // app image manifest + │ ├── 2f789..d // app image config + │ └── 3g234..f // buildpack layer ├── index.json └── oci-layout ``` +As we can see, the application image `my-app-image` has missing `blobs` because they were not provided as input and the lifecycle just **skip writing** those layers on disk. + ##### Using -oci flag in combination with --daemon or --publish flags Any combination of using multiple sources or sinks in the Lifecycle invocation of phases should throw an error to the user. For example: ```=shell > export CNB_USE_OCI=true -> /cnb/lifecycle/exporter -daemon -run-image cnb/my-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -daemon -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/exporter -oci -daemon -run-image cnb/my-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -oci -daemon -run-image cnb/my-full-stack-run:bionic my-app-image ERROR: exporting to multiples target is not allowed ``` @@ -281,10 +346,119 @@ The image look up will be done following these rules: - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - WHEN `the image points to a digest reference` - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - - Apart from the look up, the logic for each phase should remain the same + - Apart for the image look up, the logic for each phase should remain the same and the lifecycle will write the layers on disk according to the following rules: + - Any layer generated by a buildpack SHOULD always be written to disk + - Layers from `base-image` or `run-image` MAY be written on disk if they were provided + +Let's review our previous examples. + +## Examples + +In all the examples the new feature is enabled by the use of the new flag `-oci` or by setting the new environment variable `CNB_USE_OCI` to true. + +#### Analyze phase + +##### Analyzing run-image full saved on disk + +Arguments received: + + - `run-image`: `cnb/my-full-stack-run:bionic` + - `image`: `my-app-image` + +The `` is set with the default value `/oci` + +Lifecycle applies the rules for looking up the images: + - It takes the **tag reference** `cnb/my-full-stack-run:bionic` and look at path `/oci/cnb/my-full-stack-run` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. + - Lifecycle validates the `annotations` map in the **Image Manifest** contains a key-value paired `"org.opencontainers.image.ref.name" : "bionic"` + - In case of the *application image* it will look at path `/oci/my-app-image` + + Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated + +##### Analyzing run-image partial saved on disk + +Arguments received: + + - `run-image`: `cnb/my-full-partial-run:bionic` + - `image`: `my-app-image` + +The `` is set with the default value `/oci` + +Noticed the structure of the `run-image` provided + +```=shell +oci +├── cnb + └── my-partial-stack-run + ├── blobs + │ └── sha256 + │ ├── 1f59...944a // manifest + │ └── 6388...af5a // config + ├── index.json + └── oci-layout +``` + +Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/oci/cnb/my-partial-stack-run` and it determines a partial image was provided and execute the phase logic with the information from the **Image Manifest** and the **Image Config** + +##### Analyzing previous-image + +Arguments received: + +- `run-image`: `cnb/my-full-stack-run:bionic` +- `previous-image`: `bar/my-previous-app` +- `image`: `my-app-image` + +The `` is set with the default value `/oci` + +`run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/oci/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. + +##### Analyzing run-image not saved on disk + +Arguments received: + +- `run-image`: `cnb/bad-run-image` +- `image`: `my-app-image` + +The `` is set with the default value `/oci` + +In this case, Lifecycle will will look at path `/oci/bad-run-image` and because the path doesn't exists then an error must be thrown. + +##### Analyzing without run-image argument + +Arguments received: + +- `image`: `my-app-image` + +The `` is set with the default value `/oci` + +When the feature is enabled, Lifecycle requires any input image must be available on disk, because the `run-image` reference was not provided Lifecycle must thrown an error. + +#### Export phase + +##### Export to OCI using run-image full saved on disk + +Arguments received: + +- `run-image`: `cnb/my-full-stack-run:bionic` +- `image`: `my-app-image` + +The `` is set with the default value `/oci` + +Lifecycle applies the rules for looking up the images: + - It takes the **tag reference** `cnb/my-full-stack-run:bionic` and look at path `/oci/cnb/my-full-stack-run` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. + - Lifecycle validates the `annotations` map in the **Image Manifest** contains a key-value paired `"org.opencontainers.image.ref.name" : "bionic"` + - Lifecycle determines the `run-image` is a full image reference (it contains all the blobs) + - Lifecycle writes the *application image* at path `/oci/my-app-image` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format including the layers from the `run-image` + +##### Export to OCI using run-image partially saved on disk + +Arguments received: +- `run-image`: `cnb/my-partial-stack-run:bionic` +- `image`: `my-app-image` +The `` is set with the default value `/oci` +Lifecycle behaves similar to the previous example, but during the process of writing the output application image to disk it will skip the layers that are missing in the `blobs` folder at path `/oci/cnb/my-partial-stack-run` ## Proof of concept From 9eb99ad29e79ddf2cef7339d085cd3dda4d9dbb9 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 8 Nov 2022 08:23:40 -0500 Subject: [PATCH 301/372] Update text/0000-export-to-oci.md Fixes suggested by Natalie Co-authored-by: Natalie Arellano Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index be61b28dc..e7fab3b00 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -67,7 +67,7 @@ The target personas affected by this change are: - **Platform implementors**: they will have to take care of the responsibility of creating a store resource on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format and pass it through the lifecycle during the phases execution. -The process of writing any image on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be expensive in terms of hard drive space or IO operation (compressing or uncompressing layers). In order to provide flexibility for the implementation, the `analyzer` or `exporter` binaries only require the *Image Manifest* and the *Image Config* to execute their operations, based on this, we proposed the Lifecycle can be configured to work with a partial representation of the images on disk, meaning that some blobs MAY be missing (which is ok according to the [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format). The missing blobs SHOULD be those that are already available in a daemon or registry. +The process of writing any image on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format could be expensive in terms of hard drive space or IO operation (compressing or uncompressing layers). In order to provide flexibility for the implementation, the `analyzer` or `exporter` binaries only require the *Image Manifest* and the *Image Config* to execute their operations on the previous image and run image; based on this, we proposed the Lifecycle can be configured to work with a partial representation of the images on disk, meaning that some blobs MAY be missing (which is ok according to the [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format). The missing blobs SHOULD be those that are already available in a daemon or registry. Let's see some examples of the proposed behavior From e6e01fdf0cb80982718214fee3bd7169ad80b3ea Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 9 Nov 2022 17:16:12 -0500 Subject: [PATCH 302/372] renaming flags to use layout instead of oci Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 64 +++++++++++++++++++------------------- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index e7fab3b00..ba0c4ec61 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -141,7 +141,7 @@ The images named **cnb/my-full-stack-run** and **cnb/my-partial-stack-run** repr For each example case, I will present two ways of enabling the new capability: - Using an environment Variable -- Using a new `oci` flag +- Using a new `-layout` flag In any case the expected output is the same. @@ -150,12 +150,12 @@ In any case the expected output is the same. ##### Analyzing run-image full saved on disk ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -oci -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic my-app-image # expected output # analyzed.toml file with the usual `run-image.reference` @@ -165,12 +165,12 @@ In any case the expected output is the same. ##### Analyzing run-image partial saved on disk ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -oci -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -layout -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image # expected output # analyzed.toml file with the usual `run-image.reference` @@ -181,12 +181,12 @@ In any case the expected output is the same. ##### Analyzing previous-image ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic -previous-image bar/my-previous-app my-app-image # OR -> /cnb/lifecycle/analyzer -oci -run-image cnb/my-full-stack-run:bionic-previous-image bar/my-previous-app my-app-image +> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic-previous-image bar/my-previous-app my-app-image # expected output # analyzed.toml file with the usual `run-image.reference` and `previos-image.reference` @@ -195,12 +195,12 @@ In any case the expected output is the same. ##### Analyzing run-image not saved on disk ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/bad-run-image my-app-image # OR -> /cnb/lifecycle/analyzer -oci -run-image cnb/bad-run-image my-app-image +> /cnb/lifecycle/analyzer -layout -run-image cnb/bad-run-image my-app-image # expected output @@ -210,12 +210,12 @@ ERROR: the run-image could not be found at path: /oci/cnb/bad-run-image ##### Analyzing without run-image argument ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/analyzer my-app-image # OR -> /cnb/lifecycle/analyzer -oci my-app-image +> /cnb/lifecycle/analyzer -layout my-app-image # expected output @@ -227,12 +227,12 @@ ERROR: -run-image is required when OCI feature is enabled ##### Export to OCI using run-image full saved on disk ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/exporter -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/exporter -oci -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -layout -run-image cnb/my-full-stack-run:bionic my-app-image # expected output # the store folder will be updated with the application image as follows @@ -270,12 +270,12 @@ As we can see, the application image `my-app-image` contains a **full** copy of ##### Export to OCI using run-image partially saved on disk ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/exporter -run-image cnb/my-partial-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/exporter -oci -run-image cnb/my-partial-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -layout -run-image cnb/my-partial-stack-run:bionic my-app-image # expected output # the store folder will be updated with the application image as follows @@ -303,17 +303,17 @@ oci As we can see, the application image `my-app-image` has missing `blobs` because they were not provided as input and the lifecycle just **skip writing** those layers on disk. -##### Using -oci flag in combination with --daemon or --publish flags +##### Using -layout flag in combination with --daemon or --publish flags Any combination of using multiple sources or sinks in the Lifecycle invocation of phases should throw an error to the user. For example: ```=shell -> export CNB_USE_OCI=true +> export CNB_USE_OCI_LAYOUT=true > /cnb/lifecycle/exporter -daemon -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/exporter -oci -daemon -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -layout -daemon -run-image cnb/my-full-stack-run:bionic my-app-image ERROR: exporting to multiples target is not allowed ``` @@ -337,15 +337,15 @@ The following new inputs are proposed to be added to these phases | Input | Environment Variable | Default Value | Description |-------|-----------------------|---------------|-------------- - | `` | `CNB_USE_OCI` | false | Analyze or Export image from/to OCI layout format on disk | - | `` | `CNB_OCI_PATH` | /oci | Path to oci directory where the images are saved | + | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | + | `` | `CNB_OCI_LAYOUT_PATH` | /oci | Path to oci directory where the images are saved | -- WHEN `the new flag -oci or the default environment variable CNB_USE_OCI are set to true` during the phase invocation THEN the feature will be enabled. +- WHEN `the new flag -layout or the default environment variable CNB_USE_OCI_LAYOUT are set to true` during the phase invocation THEN the feature will be enabled. The image look up will be done following these rules: - WHEN `the image points to a tag reference` - - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - WHEN `the image points to a digest reference` - - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - Apart for the image look up, the logic for each phase should remain the same and the lifecycle will write the layers on disk according to the following rules: - Any layer generated by a buildpack SHOULD always be written to disk - Layers from `base-image` or `run-image` MAY be written on disk if they were provided @@ -354,7 +354,7 @@ Let's review our previous examples. ## Examples -In all the examples the new feature is enabled by the use of the new flag `-oci` or by setting the new environment variable `CNB_USE_OCI` to true. +In all the examples the new feature is enabled by the use of the new flag `-layout` or by setting the new environment variable `CNB_USE_OCI_LAYOUT` to true. #### Analyze phase @@ -365,7 +365,7 @@ Arguments received: - `run-image`: `cnb/my-full-stack-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` Lifecycle applies the rules for looking up the images: - It takes the **tag reference** `cnb/my-full-stack-run:bionic` and look at path `/oci/cnb/my-full-stack-run` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. @@ -381,7 +381,7 @@ Arguments received: - `run-image`: `cnb/my-full-partial-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` Noticed the structure of the `run-image` provided @@ -407,7 +407,7 @@ Arguments received: - `previous-image`: `bar/my-previous-app` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` `run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/oci/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. @@ -418,7 +418,7 @@ Arguments received: - `run-image`: `cnb/bad-run-image` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` In this case, Lifecycle will will look at path `/oci/bad-run-image` and because the path doesn't exists then an error must be thrown. @@ -428,7 +428,7 @@ Arguments received: - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` When the feature is enabled, Lifecycle requires any input image must be available on disk, because the `run-image` reference was not provided Lifecycle must thrown an error. @@ -441,7 +441,7 @@ Arguments received: - `run-image`: `cnb/my-full-stack-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` Lifecycle applies the rules for looking up the images: - It takes the **tag reference** `cnb/my-full-stack-run:bionic` and look at path `/oci/cnb/my-full-stack-run` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. @@ -456,7 +456,7 @@ Arguments received: - `run-image`: `cnb/my-partial-stack-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/oci` Lifecycle behaves similar to the previous example, but during the process of writing the output application image to disk it will skip the layers that are missing in the `blobs` folder at path `/oci/cnb/my-partial-stack-run` @@ -465,7 +465,7 @@ Lifecycle behaves similar to the previous example, but during the process of wri In order to validate the feasibility of the proposed feature, we developed a proof of concept with one of the most important side effects this capability can add into the project: **Removing the Daemon Support**. You can also check a recording with the demo in the following [link](https://drive.google.com/file/d/1W1125OHuyUlx88BRroUTLBfrFHhFM5A9/view?usp=sharing) As mentioned earlier, if we want to remove the daemon support in the Lifecycle, then all the responsibility to deal with it goes into the platforms implementors, that means, for example: -- Pull the require dependencies (runtime image for example), save them on disk in OCI layout format and pass it through the lifecycle using the `` parameter +- Pull the require dependencies (runtime image for example), save them on disk in OCI layout format and pass it through the lifecycle using the `` parameter - Push the application image (exported in OCI layout format) into the Daemon, because that is what users are expecting. During the proof of concept implementation I choose to use [skopeo](https://github.com/containers/skopeo) tool to solve the problem of interacting with the Daemon. The reason to do it was **simplicity** for the PoC developed but we believe this is a good subject to talk about with the community. From a38c474d4604851ae92f35538264d4f537d0f907 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 23 Jan 2023 12:37:49 -0500 Subject: [PATCH 303/372] Adding updates after the implementation Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 481 +++++++++++++++++++++++-------------- 1 file changed, 307 insertions(+), 174 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index ba0c4ec61..dbbc93eca 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -77,63 +77,42 @@ Let's see some examples of the proposed behavior A folder on disk (accessible by the lifecycle) is required to execute the feature, this new folder works as a local registry and the content must be in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -Lifecycle phases accepts arguments pointing to `image reference`, those arguments are: - -| Input | Description -|-------|------------ -| `` | Tag reference to which the app image will be written | -| `` | Image reference to be analyzed (usually the result of the previous build) | -| `` | Run image reference | - - - -Let's suppose a directory exits and it has the following structure: +Let's suppose a directory exists and it has the following structure: ```=shell -oci -├── cnb -│ ├── my-full-stack-run -│ │ ├── blobs -│ │ │ └── sha256 -│ │ │ ├── 1f59...944a // manifest -│ │ │ ├── 6388...af5a // config -│ │ │ ├── 824b...f984e -│ │ │ ├── f5f7...5b38 -│ │ │ └── 870e...f1b09 -│ │ ├── index.json -│ │ └── oci-layout -│ └── my-partial-stack-run -│ ├── blobs -│ │ └── sha256 -│ │ ├── 1f59...944a // manifest -│ │ └── 6388...af5a // config -│ ├── index.json -│ └── oci-layout -├── foo -│ └── my-cache -│ ├── blobs -│ │ └── sha256 -│ │ └── 1f591..a -│ ├── index.json -│ └── oci-layout -├── bar -│ └── my-previous-app -│ ├── blobs -│ │ └── sha256 -│ │ └── 1efg..w -│ ├── index.json -│ └── oci-layout -└── my-app-image - └── blobs - ├── sha256 - │ ├── 1bcd5..x // app image manifest - │ ├── 2f789..d // app image config - │ ├── 824b...f984e // run layer - │ ├── f5f7...5b38 // run layer - │ ├── 870e...f1b09 // run layer - │ └── 3g234..f // buildpack layer - ├── index.json - └── oci-layout +layout-repo +└── index.docker.io + ├── cnb + │ ├── my-full-stack-run:bionic + │ │ └── bionic + │ │ └── blobs + │ │ ├── sha256 + │ │ │ ├── 1f59...944a // manifest + │ │ │ ├── 6388...af5a // config + │ │ │ ├── 824b...f984e + │ │ │ ├── f5f7...5b38 + │ │ │ └── 870e...f1b09 + │ │ ├── index.json + │ │ └── oci-layout + │ └── my-partial-stack-run:bionic + │ └── bionic + │ ├── blobs + │ │ └── sha256 + │ │ ├── 1f59...944a // manifest + │ │ └── 6388...af5a // config + │ ├── index.json + │ └── oci-layout + └── bar + └── my-previous-app + └── latest + ├── blobs + │ └── sha256 + │ ├── 4bcd5..x // app image manifest + │ ├── 5f789..d // app image config + │ ├── 624b...f984e // run layer + │ └── 4g234..f // buildpack layer + ├── index.json + └── oci-layout ``` The images named **cnb/my-full-stack-run** and **cnb/my-partial-stack-run** represents the same image but the partial one has missing `blobs`, those `blobs` are the layers that are already available in the store from it came from. @@ -150,15 +129,20 @@ In any case the expected output is the same. ##### Analyzing run-image full saved on disk ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic my-app-image # OR > /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic my-app-image +``` -# expected output -# analyzed.toml file with the usual `run-image.reference` +expected analyzed.toml output + +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic" ``` @@ -171,10 +155,14 @@ In any case the expected output is the same. # OR > /cnb/lifecycle/analyzer -layout -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image +``` -# expected output -# analyzed.toml file with the usual `run-image.reference` -# there should be no change behavior in the phase even thought the image has missing blobs +expected analyzed.toml output + +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" ``` @@ -187,9 +175,19 @@ In any case the expected output is the same. # OR > /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic-previous-image bar/my-previous-app my-app-image +``` + +expected analyzed.toml output + +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + +[previous-image] + reference = "" + name = "/layout-repo/index.docker.io/bar/my-previous-app/latest" -# expected output -# analyzed.toml file with the usual `run-image.reference` and `previos-image.reference` ``` ##### Analyzing run-image not saved on disk @@ -204,7 +202,7 @@ In any case the expected output is the same. # expected output -ERROR: the run-image could not be found at path: /oci/cnb/bad-run-image +ERROR: the run-image could not be found at path: /layout-repo/index.docker.io/cnb/bad-run-image/latest ``` ##### Analyzing without run-image argument @@ -219,48 +217,54 @@ ERROR: the run-image could not be found at path: /oci/cnb/bad-run-image # expected output -ERROR: -run-image is required when OCI feature is enabled +ERROR: -run-image is required when OCI Layout feature is enabled ``` +Let's also check some examples when the export phase is executed + #### Export phase ##### Export to OCI using run-image full saved on disk ```=shell > export CNB_USE_OCI_LAYOUT=true -> /cnb/lifecycle/exporter -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter my-app-image # OR -> /cnb/lifecycle/exporter -layout -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -layout my-app-image +``` -# expected output -# the store folder will be updated with the application image as follows - -oci -├── cnb -│ └── my-full-stack-run -│ ├── blobs -│ │ └── sha256 -│ │ ├── 1f59...944a // manifest -│ │ ├── 6388...af5a // config -│ │ ├── 824b...f984e -│ │ ├── f5f7...5b38 -│ │ └── 870e...f1b09 -│ ├── index.json -│ └── oci-layout -├── // some folders omitted for simplicity -└── my-app-image - └── blobs - ├── sha256 - │ ├── 1bcd5..x // app image manifest - │ ├── 2f789..d // app image config - │ ├── 824b...f984e // run layer - │ ├── f5f7...5b38 // run layer - │ ├── 870e...f1b09 // run layer - │ └── 3g234..f // buildpack layer - ├── index.json - └── oci-layout +The output will be written into the repository folder described above and it should looks like this: + +```=shell +layout-repo +└── index.docker.io + ├── cnb + │ └── my-full-stack-run:bionic + │ └── bionic + │ └── blobs + │ ├── sha256 + │ │ ├── 1f59...944a // manifest + │ │ ├── 6388...af5a // config + │ │ ├── 824b...f984e + │ │ ├── f5f7...5b38 + │ │ └── 870e...f1b09 + │ ├── index.json + │ └── oci-layout + └── library + └── my-app-image + └── latest + ├── blobs + │ └── sha256 + │ ├── 1bcd5..x // app image manifest + │ ├── 2f789..d // app image config + │ ├── 824b...f984e // run layer + │ ├── f5f7...5b38 // run layer + │ ├── 870e...f1b09 // run layer + │ └── 3g234..f // buildpack layer + ├── index.json + └── oci-layout ``` @@ -271,33 +275,37 @@ As we can see, the application image `my-app-image` contains a **full** copy of ```=shell > export CNB_USE_OCI_LAYOUT=true -> /cnb/lifecycle/exporter -run-image cnb/my-partial-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter my-app-image # OR -> /cnb/lifecycle/exporter -layout -run-image cnb/my-partial-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -layout my-app-image +``` -# expected output -# the store folder will be updated with the application image as follows - -oci -├── cnb -│ └── my-partial-stack-run -│ ├── blobs -│ │ └── sha256 -│ │ ├── 1f59...944a // manifest -│ │ ├── 6388...af5a // config -│ ├── index.json -│ └── oci-layout -├── // some folders omitted for simplicity -└── my-app-image - └── blobs - ├── sha256 - │ ├── 1bcd5..x // app image manifest - │ ├── 2f789..d // app image config - │ └── 3g234..f // buildpack layer - ├── index.json - └── oci-layout +Expected output: + +```=shell +layout-repo +└── index.docker.io + ├── cnb + │ └── my-partial-stack-run:bionic + │ └── bionic + │ ├── blobs + │ │ └── sha256 + │ │ ├── 1f59...944a // manifest + │ │ └── 6388...af5a // config + │ ├── index.json + │ └── oci-layout + └── library + └── my-app-image + └── latest + ├── blobs + │ └── sha256 + │ ├── 1bcd5..x // app image manifest + │ ├── 2f789..d // app image config + │ └── 3g234..f // buildpack layer + ├── index.json + └── oci-layout ``` @@ -338,127 +346,230 @@ The following new inputs are proposed to be added to these phases | Input | Environment Variable | Default Value | Description |-------|-----------------------|---------------|-------------- | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | - | `` | `CNB_OCI_LAYOUT_PATH` | /oci | Path to oci directory where the images are saved | + | `` | `CNB_OCI_LAYOUT_PATH` | /layout-repo | Path to a directory where the images are saved in OCI layout format| + +## How to map an image reference into a path in the layout repository +[rules]: #rules + +In the previous examples one key element was how to translate an image reference into a path to look for in the ``, let's define those rules. + +Considering an **image reference** refers to either a tag reference or digest reference. It could have the following formats +- A tag reference refers to an identifier of form `/:` +- A digest reference refers to a content addressable identifier of form `/@:` -- WHEN `the new flag -layout or the default environment variable CNB_USE_OCI_LAYOUT are set to true` during the phase invocation THEN the feature will be enabled. The image look up will be done following these rules: - WHEN `the image points to a tag reference` - - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - WHEN `the image points to a digest reference` - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` - - Apart for the image look up, the logic for each phase should remain the same and the lifecycle will write the layers on disk according to the following rules: - - Any layer generated by a buildpack SHOULD always be written to disk - - Layers from `base-image` or `run-image` MAY be written on disk if they were provided + - WHEN `` is not provided default value will be **index.docker.io** + - IF `` is not also provided, then default value will be **library** -Let's review our previous examples. ## Examples -In all the examples the new feature is enabled by the use of the new flag `-layout` or by setting the new environment variable `CNB_USE_OCI_LAYOUT` to true. +In all the examples the new feature is enabled by the use of the new flag `-layout` or by setting +the new environment variable `CNB_USE_OCI_LAYOUT` to true. + +Let's review some of the previous examples #### Analyze phase ##### Analyzing run-image full saved on disk +Command: + +```=shell +> export CNB_USE_LAYOUT=true +> /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic my-app-image + +# OR + +> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic my-app-image +``` + Arguments received: - `run-image`: `cnb/my-full-stack-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/layout-repo` Lifecycle applies the rules for looking up the images: - - It takes the **tag reference** `cnb/my-full-stack-run:bionic` and look at path `/oci/cnb/my-full-stack-run` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. - - Lifecycle validates the `annotations` map in the **Image Manifest** contains a key-value paired `"org.opencontainers.image.ref.name" : "bionic"` - - In case of the *application image* it will look at path `/oci/my-app-image` + - It takes the **tag reference** `cnb/my-full-stack-run:bionic`, applies the conversion rules and look at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. + + - In case of the *application image* it will look at path `/layout-repo/index.docker.io/library/my-app-image/latest` - Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated +Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. A new field `Name` was added into the `analyzed.toml` that will contain the path resolved by the lifecycle, in these cases: + +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + +``` ##### Analyzing run-image partial saved on disk +Command received: + +```=shell +> export CNB_USE_OCI_LAYOUT=true +> /cnb/lifecycle/analyzer -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image + +# OR + +> /cnb/lifecycle/analyzer -layout -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image +``` + Arguments received: - `run-image`: `cnb/my-full-partial-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/layout-repo` Noticed the structure of the `run-image` provided ```=shell -oci -├── cnb - └── my-partial-stack-run - ├── blobs - │ └── sha256 - │ ├── 1f59...944a // manifest - │ └── 6388...af5a // config - ├── index.json - └── oci-layout +layout-repo +└── index.docker.io + └── cnb + └── my-partial-stack-run:bionic + └── bionic + ├── blobs + │ └── sha256 + │ ├── 1f59...944a // manifest + │ └── 6388...af5a // config + ├── index.json + └── oci-layout ``` -Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/oci/cnb/my-partial-stack-run` and it determines a partial image was provided and execute the phase logic with the information from the **Image Manifest** and the **Image Config** +Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase logic with the information from the **Image Manifest** and the **Image Config** + +The output `analyzed.toml` will also include the new `name` field with the path where the image was located. + +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + +``` ##### Analyzing previous-image +Command received: + +```=shell +> export CNB_USE_OCI_LAYOUT=true +> /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic -previous-image bar/my-previous-app my-app-image + +# OR + +> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic -previous-image bar/my-previous-app my-app-image +``` + Arguments received: - `run-image`: `cnb/my-full-stack-run:bionic` - `previous-image`: `bar/my-previous-app` - `image`: `my-app-image` -The `` is set with the default value `/oci` +The `` is set with the default value `/layout-repo` -`run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/oci/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +`run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/layout-repo/index.docker.io/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -##### Analyzing run-image not saved on disk +The `analyzed.toml` file es expected to be updated with the previous image section and the new label `name` will be also be there with the path to the `previous-image` -Arguments received: +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic" -- `run-image`: `cnb/bad-run-image` -- `image`: `my-app-image` +[previous-image] + reference = "" + name = "/layout-repo/index.docker.io/bar/my-previous-app/latest" -The `` is set with the default value `/oci` +``` -In this case, Lifecycle will will look at path `/oci/bad-run-image` and because the path doesn't exists then an error must be thrown. +Let's check how the `export` examples works on detailed -##### Analyzing without run-image argument +##### Export to OCI using run-image full saved on disk -Arguments received: +Pre-conditions: -- `image`: `my-app-image` +The following directories are accessible by the lifecycle +```=shell +/ +├── layout-repo +│ └── index.docker.io +│ └── cnb +│ └── my-full-stack-run:bionic +│ └── bionic +│ └── blobs +│ ├── sha256 +│ │ ├── 1f59...944a // manifest +│ │ ├── 6388...af5a // config +│ │ ├── 824b...f984e +│ │ ├── f5f7...5b38 +│ │ └── 870e...f1b09 +│ ├── index.json +│ └── oci-layout +└── layers + └── analyzed.tom +``` -The `` is set with the default value `/oci` +The `/layers/analyzed.toml` file contains the following data: -When the feature is enabled, Lifecycle requires any input image must be available on disk, because the `run-image` reference was not provided Lifecycle must thrown an error. +```=toml +[run-image] + reference = "" + name = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic" -#### Export phase +``` -##### Export to OCI using run-image full saved on disk +Command executed: + +```=shell +> export CNB_USE_OCI_LAYOUT=true +> /cnb/lifecycle/exporter my-app-image + +# OR + +> /cnb/lifecycle/exporter -layout my-app-image +``` Arguments received: -- `run-image`: `cnb/my-full-stack-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/oci` - -Lifecycle applies the rules for looking up the images: - - It takes the **tag reference** `cnb/my-full-stack-run:bionic` and look at path `/oci/cnb/my-full-stack-run` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. - - Lifecycle validates the `annotations` map in the **Image Manifest** contains a key-value paired `"org.opencontainers.image.ref.name" : "bionic"` - - Lifecycle determines the `run-image` is a full image reference (it contains all the blobs) - - Lifecycle writes the *application image* at path `/oci/my-app-image` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format including the layers from the `run-image` +The `` is set with the default value `/layout-repo` -##### Export to OCI using run-image partially saved on disk +Lifecycle: + - It will read the `[run-image]` section in the `analyzed.toml` and read the `name` attribute to load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. + - Lifecycle will execute the export steps and at the end of the process it will write the *application image* at path `/layout-repo/index.docker.io/library/my-app-image/latest` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format -Arguments received: -- `run-image`: `cnb/my-partial-stack-run:bionic` -- `image`: `my-app-image` +The output image will be written at: -The `` is set with the default value `/oci` +```=shell +layout-repo +└── index.docker.io + └── library + └── my-app-image + └── latest + ├── blobs + │ └── sha256 + │ ├── 1bcd5..x // app image manifest + │ ├── 2f789..d // app image config + │ ├── 824b...f984e // run layer + │ ├── f5f7...5b38 // run layer + │ ├── 870e...f1b09 // run layer + │ └── 3g234..f // buildpack layer + ├── index.json + └── oci-layout -Lifecycle behaves similar to the previous example, but during the process of writing the output application image to disk it will skip the layers that are missing in the `blobs` folder at path `/oci/cnb/my-partial-stack-run` +``` ## Proof of concept @@ -570,7 +681,29 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes - +The [Platform Interface Specification](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#inputs-5) must be updated to include the following inputs to the [Create](https://buildpacks.io/docs/concepts/components/lifecycle/create/), [Analyze](https://buildpacks.io/docs/concepts/components/lifecycle/analyze/) and [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) phases + +| Input | Environment Variable | Default Value | Description +|-------|-----------------------|---------------|-------------- +| `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | +| `` | `CNB_OCI_LAYOUT_PATH` | /layout-repo | Path to a directory where the images are saved in OCI layout format| + +Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the new `name` field + +```=toml +[image] + reference = "" + name = "" + +[run-image] + reference = "" + name = "" + +[previous-image] + reference = "" + name = "" +``` + +Where + +* `[image|run-image|previos-image].name` MUST point to the path of the image in OCI layout format following the rules describe [previously](#rules) From f498b0ac99ced52b7ca04335f41f22571d82b2de Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 23 Jan 2023 12:41:24 -0500 Subject: [PATCH 304/372] Adding updates after the implementation Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index dbbc93eca..169a414f3 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -349,7 +349,6 @@ The following new inputs are proposed to be added to these phases | `` | `CNB_OCI_LAYOUT_PATH` | /layout-repo | Path to a directory where the images are saved in OCI layout format| ## How to map an image reference into a path in the layout repository -[rules]: #rules In the previous examples one key element was how to translate an image reference into a path to look for in the ``, let's define those rules. @@ -706,4 +705,4 @@ Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform Where -* `[image|run-image|previos-image].name` MUST point to the path of the image in OCI layout format following the rules describe [previously](#rules) +* `[image|run-image|previos-image].name` MUST point to the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) From 5d9b008f3f6db79993cbcf0846f52aee8bb2236f Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 23 Jan 2023 12:56:53 -0500 Subject: [PATCH 305/372] Open questions Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 169a414f3..97690fd39 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -672,9 +672,15 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is # Unresolved Questions [unresolved-questions]: #unresolved-questions - # Spec. Changes (OPTIONAL) From 6fc05ff497934e1e2baa9a56900e2030394fa8d3 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 23 Jan 2023 15:22:25 -0500 Subject: [PATCH 306/372] Unresolved question about exporting to tarball Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 97690fd39..fd2b1e3fb 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -676,6 +676,7 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is - Tools like [umoci](https://umo.ci/) used to create a runtime bundle from an image in OCI layout format, requires the [annotation](https://github.com/opencontainers/image-spec/blob/main/annotations.md#pre-defined-annotation-keys) `org.opencontainers.image.ref.name` to be present. Also tools like [skopeo](https://github.com/containers/skopeo) when an image is `copy` in oci format the annotation is included. We are not adding the annotation as part of the Buildpacks Specification, but in this case this could make our output incompatible with some other tooling. - Depending on the previous question, the rules for mapping an image reference into a path could change, also validations that the Lifecycle could do to verify the `org.opencontainers.image.ref.name` against the value provided in the image reference. + - Exporting to a tarball can be handle on this RFC or a new one must be created? - What parts of the design do you expect to be resolved through implementation of the feature? - Handle symbolic links to the blobs in the `` repository, this could be more efficient on hard drive space From 40cd649d67b3611e18160744821f5baa665a54aa Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 24 Jan 2023 09:39:52 -0500 Subject: [PATCH 307/372] updating the rules to map image ref to path Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index fd2b1e3fb..82724153f 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -353,14 +353,14 @@ The following new inputs are proposed to be added to these phases In the previous examples one key element was how to translate an image reference into a path to look for in the ``, let's define those rules. Considering an **image reference** refers to either a tag reference or digest reference. It could have the following formats -- A tag reference refers to an identifier of form `/:` -- A digest reference refers to a content addressable identifier of form `/@:` +- A tag reference refers to an identifier of form `//:` +- A digest reference refers to a content addressable identifier of form `//@:` The image look up will be done following these rules: - WHEN `the image points to a tag reference` - - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `////` - WHEN `the image points to a digest reference` - - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `///` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `/////` - WHEN `` is not provided default value will be **index.docker.io** - IF `` is not also provided, then default value will be **library** From 1771f9ff51157c185d6eaa6eb5eb341d174ce31d Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 30 Jan 2023 11:55:33 -0500 Subject: [PATCH 308/372] Removing the Name field from the analyzed.toml Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 48 ++++++++++++++++---------------------- 1 file changed, 20 insertions(+), 28 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 82724153f..cfbc869fb 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -141,8 +141,7 @@ expected analyzed.toml output ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -161,8 +160,7 @@ expected analyzed.toml output ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -181,12 +179,10 @@ expected analyzed.toml output ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" [previous-image] - reference = "" - name = "/layout-repo/index.docker.io/bar/my-previous-app/latest" + reference = "/layout-repo/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" ``` @@ -399,12 +395,11 @@ Lifecycle applies the rules for looking up the images: - In case of the *application image* it will look at path `/layout-repo/index.docker.io/library/my-app-image/latest` -Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. A new field `Name` was added into the `analyzed.toml` that will contain the path resolved by the lifecycle, in these cases: +Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. The `run-image.reference` added into the `analyzed.toml` will contain the path resolved by the lifecycle plus the digest reference to the image with the following format `[path]@[digest]`. In case of this example, it will look like this: ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -444,14 +439,13 @@ layout-repo └── oci-layout ``` -Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase logic with the information from the **Image Manifest** and the **Image Config** +Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase with the information from the **Image Manifest** and the **Image Config** -The output `analyzed.toml` will also include the new `name` field with the path where the image was located. +The output `analyzed.toml` will also include the new `run-image.reference` field the path and the digest of the run image. ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -478,16 +472,14 @@ The `` is set with the default value `/layout-repo` `run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/layout-repo/index.docker.io/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -The `analyzed.toml` file es expected to be updated with the previous image section and the new label `name` will be also be there with the path to the `previous-image` +The `analyzed.toml` file es expected to be updated with the `previous-image.reference` containing the path and the digest of the `previous-image` ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" [previous-image] - reference = "" - name = "/layout-repo/index.docker.io/bar/my-previous-app/latest" + reference = "/layout-repo/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" ``` @@ -522,8 +514,7 @@ The `/layers/analyzed.toml` file contains the following data: ```=toml [run-image] - reference = "" - name = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -545,7 +536,8 @@ Arguments received: The `` is set with the default value `/layout-repo` Lifecycle: - - It will read the `[run-image]` section in the `analyzed.toml` and read the `name` attribute to load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. + - It will read the `[run-image]` section in the `analyzed.toml`, it will parse `reference` attribute using the `@` separator and load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. + - Lifecycle could also validate the digest of the image loaded is the same as the one established by the `reference`. - Lifecycle will execute the export steps and at the end of the process it will write the *application image* at path `/layout-repo/index.docker.io/library/my-app-image/latest` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format @@ -694,22 +686,22 @@ The [Platform Interface Specification](https://github.com/buildpacks/spec/blob/p | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | | `` | `CNB_OCI_LAYOUT_PATH` | /layout-repo | Path to a directory where the images are saved in OCI layout format| -Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the new `name` field +Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the `reference` format in case of layout is being used. ```=toml [image] reference = "" - name = "" [run-image] reference = "" - name = "" [previous-image] reference = "" - name = "" ``` Where -* `[image|run-image|previos-image].name` MUST point to the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) +- `[image|run-image|previos-image].reference` MUST be either a digest reference to an image in an OCI registry or the ID of an image in a docker daemon. + - In case an image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format is being used, it will also include the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) + - The format MUST be as follows: `[path]@[digest]` + From 35bd3e9513454b17c5f29bb7fb5564fd17cff0b8 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 6 Feb 2023 12:49:10 -0500 Subject: [PATCH 309/372] removing the layout-repo flag Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 78 +++++++++++++++----------------------- 1 file changed, 31 insertions(+), 47 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index cfbc869fb..06e7247b8 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -75,13 +75,12 @@ Let's see some examples of the proposed behavior ### Requirements -A folder on disk (accessible by the lifecycle) is required to execute the feature, this new folder works as a local registry and the content must be in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +Lifecycle will converts image references into local paths following define [rules](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) and the content must be in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -Let's suppose a directory exists and it has the following structure: +Let's suppose a *platform implementor* creates directories with the following structure: ```=shell -layout-repo -└── index.docker.io +index.docker.io ├── cnb │ ├── my-full-stack-run:bionic │ │ └── bionic @@ -141,7 +140,7 @@ expected analyzed.toml output ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -160,7 +159,7 @@ expected analyzed.toml output ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-partial-stack-run@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -179,10 +178,10 @@ expected analyzed.toml output ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" [previous-image] - reference = "/layout-repo/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" + reference = "/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" ``` @@ -198,7 +197,7 @@ expected analyzed.toml output # expected output -ERROR: the run-image could not be found at path: /layout-repo/index.docker.io/cnb/bad-run-image/latest +ERROR: the run-image could not be found at path: /index.docker.io/cnb/bad-run-image/latest ``` ##### Analyzing without run-image argument @@ -234,8 +233,7 @@ Let's also check some examples when the export phase is executed The output will be written into the repository folder described above and it should looks like this: ```=shell -layout-repo -└── index.docker.io +index.docker.io ├── cnb │ └── my-full-stack-run:bionic │ └── bionic @@ -281,8 +279,7 @@ As we can see, the application image `my-app-image` contains a **full** copy of Expected output: ```=shell -layout-repo -└── index.docker.io +index.docker.io ├── cnb │ └── my-partial-stack-run:bionic │ └── bionic @@ -341,22 +338,21 @@ The following new inputs are proposed to be added to these phases | Input | Environment Variable | Default Value | Description |-------|-----------------------|---------------|-------------- - | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | - | `` | `CNB_OCI_LAYOUT_PATH` | /layout-repo | Path to a directory where the images are saved in OCI layout format| + | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | ## How to map an image reference into a path in the layout repository -In the previous examples one key element was how to translate an image reference into a path to look for in the ``, let's define those rules. +In the previous examples one key element was how to translate an image reference into a path, let's define those rules. Considering an **image reference** refers to either a tag reference or digest reference. It could have the following formats -- A tag reference refers to an identifier of form `//:` +- A name reference refers to an identifier of form `//:` - A digest reference refers to a content addressable identifier of form `//@:` The image look up will be done following these rules: - - WHEN `the image points to a tag reference` - - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `////` + - WHEN `the image points to a name reference` + - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `////` - WHEN `the image points to a digest reference` - - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `/////` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `/////` - WHEN `` is not provided default value will be **index.docker.io** - IF `` is not also provided, then default value will be **library** @@ -388,18 +384,16 @@ Arguments received: - `run-image`: `cnb/my-full-stack-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/layout-repo` - Lifecycle applies the rules for looking up the images: - - It takes the **tag reference** `cnb/my-full-stack-run:bionic`, applies the conversion rules and look at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. + - It takes the **tag reference** `cnb/my-full-stack-run:bionic`, applies the conversion rules and look at path `/index.docker.io/cnb/my-full-stack-run/bionic` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. - - In case of the *application image* it will look at path `/layout-repo/index.docker.io/library/my-app-image/latest` + - In case of the *application image* it will look at path `/index.docker.io/library/my-app-image/latest` Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. The `run-image.reference` added into the `analyzed.toml` will contain the path resolved by the lifecycle plus the digest reference to the image with the following format `[path]@[digest]`. In case of this example, it will look like this: ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -421,13 +415,10 @@ Arguments received: - `run-image`: `cnb/my-full-partial-run:bionic` - `image`: `my-app-image` -The `` is set with the default value `/layout-repo` - Noticed the structure of the `run-image` provided ```=shell -layout-repo -└── index.docker.io +index.docker.io └── cnb └── my-partial-stack-run:bionic └── bionic @@ -439,13 +430,13 @@ layout-repo └── oci-layout ``` -Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase with the information from the **Image Manifest** and the **Image Config** +Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase with the information from the **Image Manifest** and the **Image Config** The output `analyzed.toml` will also include the new `run-image.reference` field the path and the digest of the run image. ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -468,18 +459,16 @@ Arguments received: - `previous-image`: `bar/my-previous-app` - `image`: `my-app-image` -The `` is set with the default value `/layout-repo` - -`run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/layout-repo/index.docker.io/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +`run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/index.docker.io/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. The `analyzed.toml` file es expected to be updated with the `previous-image.reference` containing the path and the digest of the `previous-image` ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" [previous-image] - reference = "/layout-repo/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" + reference = "/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" ``` @@ -492,8 +481,7 @@ Pre-conditions: The following directories are accessible by the lifecycle ```=shell / -├── layout-repo -│ └── index.docker.io +└── index.docker.io │ └── cnb │ └── my-full-stack-run:bionic │ └── bionic @@ -514,7 +502,7 @@ The `/layers/analyzed.toml` file contains the following data: ```=toml [run-image] - reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -533,19 +521,16 @@ Arguments received: - `image`: `my-app-image` -The `` is set with the default value `/layout-repo` - Lifecycle: - - It will read the `[run-image]` section in the `analyzed.toml`, it will parse `reference` attribute using the `@` separator and load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. + - It will read the `[run-image]` section in the `analyzed.toml`, it will parse `reference` attribute using the `@` separator and load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/index.docker.io/cnb/my-full-stack-run/bionic`. - Lifecycle could also validate the digest of the image loaded is the same as the one established by the `reference`. - - Lifecycle will execute the export steps and at the end of the process it will write the *application image* at path `/layout-repo/index.docker.io/library/my-app-image/latest` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format + - Lifecycle will execute the export steps and at the end of the process it will write the *application image* at path `/index.docker.io/library/my-app-image/latest` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format The output image will be written at: ```=shell -layout-repo -└── index.docker.io +index.docker.io └── library └── my-app-image └── latest @@ -671,7 +656,7 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is - Exporting to a tarball can be handle on this RFC or a new one must be created? - What parts of the design do you expect to be resolved through implementation of the feature? - - Handle symbolic links to the blobs in the `` repository, this could be more efficient on hard drive space + - Handle symbolic links to the blobs in the repository, this could be more efficient on hard drive space @@ -684,7 +669,6 @@ The [Platform Interface Specification](https://github.com/buildpacks/spec/blob/p | Input | Environment Variable | Default Value | Description |-------|-----------------------|---------------|-------------- | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | -| `` | `CNB_OCI_LAYOUT_PATH` | /layout-repo | Path to a directory where the images are saved in OCI layout format| Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the `reference` format in case of layout is being used. From eba5b7e9a9cc17fd7012df975edf4989bee951df Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 8 Feb 2023 17:26:35 -0500 Subject: [PATCH 310/372] Revert "Removing the Name field from the analyzed.toml" This reverts commit 9e5c2d0ba198112d0e808b4f04a25937287e48cc. Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 48 +++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 06e7247b8..53f70e8a6 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -140,7 +140,7 @@ expected analyzed.toml output ```=toml [run-image] - reference = "/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -159,7 +159,7 @@ expected analyzed.toml output ```=toml [run-image] - reference = "/index.docker.io/cnb/my-partial-stack-run@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -178,10 +178,10 @@ expected analyzed.toml output ```=toml [run-image] - reference = "/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" [previous-image] - reference = "/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" + reference = "/layout-repo/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" ``` @@ -336,8 +336,8 @@ Notice that we are relying on the OCI format Specification to expose the data fo The following new inputs are proposed to be added to these phases - | Input | Environment Variable | Default Value | Description - |-------|-----------------------|---------------|-------------- + | Input | Environment Variable | Default Value | Description | + |-------|-----------------------|---------------|-------------| | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | ## How to map an image reference into a path in the layout repository @@ -389,12 +389,11 @@ Lifecycle applies the rules for looking up the images: - In case of the *application image* it will look at path `/index.docker.io/library/my-app-image/latest` -Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. The `run-image.reference` added into the `analyzed.toml` will contain the path resolved by the lifecycle plus the digest reference to the image with the following format `[path]@[digest]`. In case of this example, it will look like this: +Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. A new field `Name` was added into the `analyzed.toml` that will contain the path resolved by the lifecycle, in these cases: ```=toml [run-image] - reference = "/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" - + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` ##### Analyzing run-image partial saved on disk @@ -430,14 +429,12 @@ index.docker.io └── oci-layout ``` -Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase with the information from the **Image Manifest** and the **Image Config** - -The output `analyzed.toml` will also include the new `run-image.reference` field the path and the digest of the run image. +Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase with the information from the **Image Manifest** and the **Image Config** +The output `analyzed.toml` will also include the new `name` field with the path where the image was located. ```=toml [run-image] - reference = "/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" - + reference = "/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` ##### Analyzing previous-image @@ -461,14 +458,14 @@ Arguments received: `run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/index.docker.io/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -The `analyzed.toml` file es expected to be updated with the `previous-image.reference` containing the path and the digest of the `previous-image` +The `analyzed.toml` file es expected to be updated with the previous image section and the new label `name` will be also be there with the path to the `previous-image` ```=toml [run-image] - reference = "/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" [previous-image] - reference = "/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" + reference = "/layout-repo/index.docker.io/bar/my-previous-app/latest@sha256:aa0cf7fc8f161bdb96166c1644174affacd70d17f372373ca72c8e91116e2d43" ``` @@ -502,7 +499,7 @@ The `/layers/analyzed.toml` file contains the following data: ```=toml [run-image] - reference = "/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" + reference = "/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic@sha256:fab3bb83de466ed29d7e9dcfdbee5b5fb2ff90e91bc849af85b261b4c2062a7a" ``` @@ -522,9 +519,9 @@ Arguments received: - `image`: `my-app-image` Lifecycle: - - It will read the `[run-image]` section in the `analyzed.toml`, it will parse `reference` attribute using the `@` separator and load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/index.docker.io/cnb/my-full-stack-run/bionic`. + - It will read the `[run-image]` section in the `analyzed.toml`, it will parse `reference` attribute using the `@` separator and load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. - Lifecycle could also validate the digest of the image loaded is the same as the one established by the `reference`. - - Lifecycle will execute the export steps and at the end of the process it will write the *application image* at path `/index.docker.io/library/my-app-image/latest` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format + - Lifecycle will execute the export steps and at the end of the process it will write the *application image* at path `/layout-repo/index.docker.io/library/my-app-image/latest` in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format The output image will be written at: @@ -666,11 +663,11 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is The [Platform Interface Specification](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#inputs-5) must be updated to include the following inputs to the [Create](https://buildpacks.io/docs/concepts/components/lifecycle/create/), [Analyze](https://buildpacks.io/docs/concepts/components/lifecycle/analyze/) and [Export](https://buildpacks.io/docs/concepts/components/lifecycle/export/) phases -| Input | Environment Variable | Default Value | Description -|-------|-----------------------|---------------|-------------- +| Input | Environment Variable | Default Value | Description| +|-------|-----------------------|---------------|------------| | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | -Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the `reference` format in case of layout is being used. +Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the new `name` field ```=toml [image] @@ -685,7 +682,4 @@ Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform Where -- `[image|run-image|previos-image].reference` MUST be either a digest reference to an image in an OCI registry or the ID of an image in a docker daemon. - - In case an image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format is being used, it will also include the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) - - The format MUST be as follows: `[path]@[digest]` - +* `[image|run-image|previos-image].name` MUST point to the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) From 1086f453a927260743f7d4a70a0e9e3c591d4e2e Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 8 Feb 2023 18:27:57 -0500 Subject: [PATCH 311/372] adding Natalie's feedback Signed-off-by: Juan Bustamante Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 131 ++++++++++++++++++++++++------------- 1 file changed, 87 insertions(+), 44 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 53f70e8a6..3cb550a98 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -77,10 +77,11 @@ Let's see some examples of the proposed behavior Lifecycle will converts image references into local paths following define [rules](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) and the content must be in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -Let's suppose a *platform implementor* creates directories with the following structure: +Let's suppose a *platform implementor* creates a directory with the following structure: ```=shell -index.docker.io +layout-repo +└── index.docker.io ├── cnb │ ├── my-full-stack-run:bionic │ │ └── bionic @@ -118,8 +119,8 @@ The images named **cnb/my-full-stack-run** and **cnb/my-partial-stack-run** repr For each example case, I will present two ways of enabling the new capability: -- Using an environment Variable -- Using a new `-layout` flag +- Using an environment variables +- Using the new `-layout` and `layout-dir` flags In any case the expected output is the same. @@ -129,11 +130,12 @@ In any case the expected output is the same. ```=shell > export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -layout -layout-dir /layout-repo -run-image cnb/my-full-stack-run:bionic my-app-image ``` expected analyzed.toml output @@ -147,12 +149,13 @@ expected analyzed.toml output ##### Analyzing run-image partial saved on disk ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/analyzer -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -layout -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -layout -layout-dir /layout-repo -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image ``` expected analyzed.toml output @@ -166,12 +169,13 @@ expected analyzed.toml output ##### Analyzing previous-image ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic -previous-image bar/my-previous-app my-app-image # OR -> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic-previous-image bar/my-previous-app my-app-image +> /cnb/lifecycle/analyzer -layout -layout-dir /layout-repo -run-image cnb/my-full-stack-run:bionic-previous-image bar/my-previous-app my-app-image ``` expected analyzed.toml output @@ -188,33 +192,48 @@ expected analyzed.toml output ##### Analyzing run-image not saved on disk ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/analyzer -run-image cnb/bad-run-image my-app-image # OR -> /cnb/lifecycle/analyzer -layout -run-image cnb/bad-run-image my-app-image +> /cnb/lifecycle/analyzer -layout -layout-dir /layout-repo -run-image cnb/bad-run-image my-app-image # expected output -ERROR: the run-image could not be found at path: /index.docker.io/cnb/bad-run-image/latest +ERROR: the run-image could not be found at path: /layout-repo/index.docker.io/cnb/bad-run-image/latest ``` ##### Analyzing without run-image argument ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/analyzer my-app-image # OR -> /cnb/lifecycle/analyzer -layout my-app-image +> /cnb/lifecycle/analyzer -layout -layout-dir /layout-repo my-app-image # expected output ERROR: -run-image is required when OCI Layout feature is enabled ``` +```=shell +> export CNB_USE_LAYOUT=true +> /cnb/lifecycle/analyzer -run-image cnb/bad-run-image my-app-image + +# OR + +> /cnb/lifecycle/analyzer -layout -run-image cnb/bad-run-image my-app-image + +# expected output + +ERROR: defining a layout directory is required when OCI Layout feature is enabled. Use -layout-dir flag or CNB_LAYOUT_DIR environment variable +``` + Let's also check some examples when the export phase is executed #### Export phase @@ -222,18 +241,20 @@ Let's also check some examples when the export phase is executed ##### Export to OCI using run-image full saved on disk ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/exporter my-app-image # OR -> /cnb/lifecycle/exporter -layout my-app-image +> /cnb/lifecycle/exporter -layout -layout-dir /layout-repo my-app-image ``` The output will be written into the repository folder described above and it should looks like this: ```=shell -index.docker.io +layout-repo +└── index.docker.io ├── cnb │ └── my-full-stack-run:bionic │ └── bionic @@ -268,18 +289,20 @@ As we can see, the application image `my-app-image` contains a **full** copy of ##### Export to OCI using run-image partially saved on disk ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/exporter my-app-image # OR -> /cnb/lifecycle/exporter -layout my-app-image +> /cnb/lifecycle/exporter -layout -layout-dir /layout-repo my-app-image ``` Expected output: ```=shell -index.docker.io +layout-repo +└── index.docker.io ├── cnb │ └── my-partial-stack-run:bionic │ └── bionic @@ -309,12 +332,13 @@ As we can see, the application image `my-app-image` has missing `blobs` because Any combination of using multiple sources or sinks in the Lifecycle invocation of phases should throw an error to the user. For example: ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/exporter -daemon -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/exporter -layout -daemon -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/exporter -layout -layout-dir /layout-repo -daemon -run-image cnb/my-full-stack-run:bionic my-app-image ERROR: exporting to multiples target is not allowed ``` @@ -336,9 +360,10 @@ Notice that we are relying on the OCI format Specification to expose the data fo The following new inputs are proposed to be added to these phases - | Input | Environment Variable | Default Value | Description | - |-------|-----------------------|---------------|-------------| - | `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | + | Input | Environment Variable | Default Value | Description + |-------|-----------------------|---------------|-------------- + | `` | `CNB_USE_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | + | `` | `CNB_LAYOUT_DIR` | | Path to a directory where the images are saved in OCI layout format| ## How to map an image reference into a path in the layout repository @@ -360,9 +385,9 @@ The image look up will be done following these rules: ## Examples In all the examples the new feature is enabled by the use of the new flag `-layout` or by setting -the new environment variable `CNB_USE_OCI_LAYOUT` to true. +the new environment variable `CNB_USE_LAYOUT` to true. -Let's review some of the previous examples +Let's review some previous examples #### Analyze phase @@ -372,11 +397,12 @@ Command: ```=shell > export CNB_USE_LAYOUT=true +> export CNB_LAYOUT_DIR=/layout-repo > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic my-app-image # OR -> /cnb/lifecycle/analyzer -layout -run-image cnb/my-full-stack-run:bionic my-app-image +> /cnb/lifecycle/analyzer -layout -layout-dir /layout-repo -run-image cnb/my-full-stack-run:bionic my-app-image ``` Arguments received: @@ -384,12 +410,15 @@ Arguments received: - `run-image`: `cnb/my-full-stack-run:bionic` - `image`: `my-app-image` -Lifecycle applies the rules for looking up the images: - - It takes the **tag reference** `cnb/my-full-stack-run:bionic`, applies the conversion rules and look at path `/index.docker.io/cnb/my-full-stack-run/bionic` for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. +The `` is set with the value `/layout-repo` - - In case of the *application image* it will look at path `/index.docker.io/library/my-app-image/latest` +Lifecycle applies the rules for looking up the images: + - It takes the **tag reference** `cnb/my-full-stack-run:bionic`, applies the conversion rules and gets `/index.docker.io/cnb/my-full-stack-run/bionic` + - It will append the `` at the beginning, getting the following path: `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic` + - It will look for an image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. + - In case of the *application image* it will look at path `/layout-repo/index.docker.io/library/my-app-image/latest` -Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. A new field `Name` was added into the `analyzed.toml` that will contain the path resolved by the lifecycle, in these cases: +Because both images are found, the phase is executed as usual and the `analyzed.toml` file will be updated. The `run-image.reference` added into the `analyzed.toml` will contain the path resolved by the lifecycle plus the digest reference to the image with the following format `[path]@[digest]`. In case of this example, it will look like this: ```=toml [run-image] @@ -401,7 +430,7 @@ Because both images are found, the phase is executed as usual and the `analyzed. Command received: ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/cnb/my-partial-stack-run:bionic my-app-image # OR @@ -414,10 +443,13 @@ Arguments received: - `run-image`: `cnb/my-full-partial-run:bionic` - `image`: `my-app-image` +The `` is set with the default value `/layout-repo` + Noticed the structure of the `run-image` provided ```=shell -index.docker.io +layout-repo +└── index.docker.io └── cnb └── my-partial-stack-run:bionic └── bionic @@ -430,7 +462,8 @@ index.docker.io ``` Similar to the previous example, Lifecycle applies the rules for looking up the images and look at path `/layout-repo/index.docker.io/cnb/my-partial-stack-run/bionic` and it determines a partial image was provided and execute the phase with the information from the **Image Manifest** and the **Image Config** -The output `analyzed.toml` will also include the new `name` field with the path where the image was located. + +The output `analyzed.toml` will also include the new `run-image.reference` field the path and the digest of the run image. ```=toml [run-image] @@ -442,7 +475,7 @@ The output `analyzed.toml` will also include the new `name` field with the path Command received: ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/my-full-stack-run:bionic -previous-image bar/my-previous-app my-app-image # OR @@ -456,9 +489,11 @@ Arguments received: - `previous-image`: `bar/my-previous-app` - `image`: `my-app-image` +The `` is set with the default value `/layout-repo` + `run-image` and `image` arguments are treated in the same way as previous examples, and for `previous-image` argument the looking up images rules are applied and Lifecycle will look at path `/index.docker.io/bar/my-previous-app` for a image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format. -The `analyzed.toml` file es expected to be updated with the previous image section and the new label `name` will be also be there with the path to the `previous-image` +The `analyzed.toml` file es expected to be updated with the `previous-image.reference` containing the path and the digest of the `previous-image` ```=toml [run-image] @@ -478,7 +513,8 @@ Pre-conditions: The following directories are accessible by the lifecycle ```=shell / -└── index.docker.io +├── layout-repo +│ └── index.docker.io │ └── cnb │ └── my-full-stack-run:bionic │ └── bionic @@ -506,7 +542,7 @@ The `/layers/analyzed.toml` file contains the following data: Command executed: ```=shell -> export CNB_USE_OCI_LAYOUT=true +> export CNB_USE_LAYOUT=true > /cnb/lifecycle/exporter my-app-image # OR @@ -518,6 +554,8 @@ Arguments received: - `image`: `my-app-image` +The `` is set with the default value `/layout-repo` + Lifecycle: - It will read the `[run-image]` section in the `analyzed.toml`, it will parse `reference` attribute using the `@` separator and load the `run-image` image saved on disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at path `/layout-repo/index.docker.io/cnb/my-full-stack-run/bionic`. - Lifecycle could also validate the digest of the image loaded is the same as the one established by the `reference`. @@ -527,7 +565,8 @@ Lifecycle: The output image will be written at: ```=shell -index.docker.io +layout-repo +└── index.docker.io └── library └── my-app-image └── latest @@ -653,7 +692,7 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is - Exporting to a tarball can be handle on this RFC or a new one must be created? - What parts of the design do you expect to be resolved through implementation of the feature? - - Handle symbolic links to the blobs in the repository, this could be more efficient on hard drive space + - Handle symbolic links to the blobs in the `` repository, this could be more efficient on hard drive space @@ -665,9 +704,10 @@ The [Platform Interface Specification](https://github.com/buildpacks/spec/blob/p | Input | Environment Variable | Default Value | Description| |-------|-----------------------|---------------|------------| -| `` | `CNB_USE_OCI_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | +| `` | `CNB_USE_LAYOUT` | false | Enables the capability of resolving image from/to in OCI layout format on disk | +| `` | `CNB_LAYOUT_DIR` | | Path to a directory where the images are saved in OCI layout format| -Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the new `name` field +Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform/0.11/platform.md#analyzedtoml-toml) will be updated to include the `reference` format in case of layout is being used. ```=toml [image] @@ -682,4 +722,7 @@ Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform Where -* `[image|run-image|previos-image].name` MUST point to the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) +- `[image|run-image|previos-image].reference` MUST be either a digest reference to an image in an OCI registry or the ID of an image in a docker daemon. + - In case an image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format is being used, it will also include the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) + - The format MUST be as follows: `[path]@[digest]` + From c1f4f63ef7f3a4348de6a63c2c7bb7533df679c3 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Tue, 14 Feb 2023 18:28:46 -0600 Subject: [PATCH 312/372] RFC 0118 [#272] Signed-off-by: Terence Lee --- text/{0000-2023H1-roadmap.md => 0118-2023H1-roadmap.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-2023H1-roadmap.md => 0118-2023H1-roadmap.md} (98%) diff --git a/text/0000-2023H1-roadmap.md b/text/0118-2023H1-roadmap.md similarity index 98% rename from text/0000-2023H1-roadmap.md rename to text/0118-2023H1-roadmap.md index 2c5538926..38b2241d3 100644 --- a/text/0000-2023H1-roadmap.md +++ b/text/0118-2023H1-roadmap.md @@ -3,10 +3,10 @@ - Name: 2023H1 Roadmap - Start Date: 2023-01-24 - Author(s): hone -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [#272](https://github.com///pull/272) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 326e7b16bb0dd1bec07c215ed172d6f6381eb550 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Tue, 14 Feb 2023 18:33:14 -0600 Subject: [PATCH 313/372] fix RFC link Signed-off-by: Terence Lee --- text/0118-2023H1-roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0118-2023H1-roadmap.md b/text/0118-2023H1-roadmap.md index 38b2241d3..ca582b201 100644 --- a/text/0118-2023H1-roadmap.md +++ b/text/0118-2023H1-roadmap.md @@ -4,7 +4,7 @@ - Start Date: 2023-01-24 - Author(s): hone - Status: Approved -- RFC Pull Request: [#272](https://github.com///pull/272) +- RFC Pull Request: [#272](https://github.com/buildpacks/rfcs/pull/272) - CNB Pull Request: (leave blank) - CNB Issue: - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) From b09fea05ebd6c039e5c2acb9cdfd03202f3e796d Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 15 Feb 2023 11:58:46 -0500 Subject: [PATCH 314/372] adding layout-dir to some missing places Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 3cb550a98..9bcd5fccd 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -221,6 +221,8 @@ ERROR: the run-image could not be found at path: /layout-repo/index.docker.io/cn ERROR: -run-image is required when OCI Layout feature is enabled ``` +##### Analyzing without layout-dir argument + ```=shell > export CNB_USE_LAYOUT=true > /cnb/lifecycle/analyzer -run-image cnb/bad-run-image my-app-image @@ -375,9 +377,9 @@ Considering an **image reference** refers to either a tag reference or digest re The image look up will be done following these rules: - WHEN `the image points to a name reference` - - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `////` + - Lifecycle will load/save the image from/to disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `////` - WHEN `the image points to a digest reference` - - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `/////` + - Lifecycle will load the image from disk in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format at `/////` - WHEN `` is not provided default value will be **index.docker.io** - IF `` is not also provided, then default value will be **library** From 76d034f4c6ff2470e85b38d2310d343f97d950f2 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 15 Feb 2023 16:37:34 -0500 Subject: [PATCH 315/372] updating the unresolved questions Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index 9bcd5fccd..c96850128 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -690,12 +690,13 @@ I think, this PoC demonstrate that adding the exporting to OCI layout format is - What parts of the design do you expect to be resolved before this gets merged? - Tools like [umoci](https://umo.ci/) used to create a runtime bundle from an image in OCI layout format, requires the [annotation](https://github.com/opencontainers/image-spec/blob/main/annotations.md#pre-defined-annotation-keys) `org.opencontainers.image.ref.name` to be present. Also tools like [skopeo](https://github.com/containers/skopeo) when an image is `copy` in oci format the annotation is included. We are not adding the annotation as part of the Buildpacks Specification, but in this case this could make our output incompatible with some other tooling. - - Depending on the previous question, the rules for mapping an image reference into a path could change, also validations that the Lifecycle could do to verify the `org.opencontainers.image.ref.name` against the value provided in the image reference. - - Exporting to a tarball can be handle on this RFC or a new one must be created? + - **Answer:** we agreed on adding `org.opencontainers.image.ref.name` annotation + - Exporting to a tarball can be handled on this RFC or a new one must be created? + - **Answer:** this can be handled on a different RFC - What parts of the design do you expect to be resolved through implementation of the feature? - Handle symbolic links to the blobs in the `` repository, this could be more efficient on hard drive space - + - **Answer:** this can be handled on the implementation side From 77c9fcce82e1504d29d119e6444e8d5d001e3ddd Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 15 Feb 2023 16:44:18 -0500 Subject: [PATCH 316/372] updating the spec changes Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index c96850128..dc4e70dbd 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -725,7 +725,8 @@ Also the `analyzed.toml` [file](https://github.com/buildpacks/spec/blob/platform Where -- `[image|run-image|previos-image].reference` MUST be either a digest reference to an image in an OCI registry or the ID of an image in a docker daemon. - - In case an image in [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) format is being used, it will also include the path of the image in OCI layout format following the rules describe [previously](#how-to-map-an-image-reference-into-a-path-in-the-layout-repository) - - The format MUST be as follows: `[path]@[digest]` +- `[image|run-image|previos-image].reference` MUST be either: + - A digest reference to an image in an OCI registry + - The ID of an image in a docker daemon + - The path to an image in OCI layout format From 8beb9e83ba5efc77bb26985dbd2a2daccd6199ec Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Fri, 17 Feb 2023 14:47:56 -0500 Subject: [PATCH 317/372] Update 0000-export-to-oci.md Fixing error message according to the implementation Signed-off-by: Juan Bustamante --- text/0000-export-to-oci.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-export-to-oci.md b/text/0000-export-to-oci.md index dc4e70dbd..d1b806f22 100644 --- a/text/0000-export-to-oci.md +++ b/text/0000-export-to-oci.md @@ -342,7 +342,7 @@ Any combination of using multiple sources or sinks in the Lifecycle invocation o > /cnb/lifecycle/exporter -layout -layout-dir /layout-repo -daemon -run-image cnb/my-full-stack-run:bionic my-app-image -ERROR: exporting to multiples target is not allowed +ERROR: exporting to multiple targets is unsupported ``` # How it Works From b533d041d70fcfe53be251b9710d0781edb45d5c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 20 Feb 2023 01:05:56 +0000 Subject: [PATCH 318/372] Bump kentaro-m/auto-assign-action from 1.2.4 to 1.2.5 Bumps [kentaro-m/auto-assign-action](https://github.com/kentaro-m/auto-assign-action) from 1.2.4 to 1.2.5. - [Release notes](https://github.com/kentaro-m/auto-assign-action/releases) - [Commits](https://github.com/kentaro-m/auto-assign-action/compare/v1.2.4...v1.2.5) --- updated-dependencies: - dependency-name: kentaro-m/auto-assign-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- .github/workflows/auto-assign-maintainer.yml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/auto-assign-maintainer.yml b/.github/workflows/auto-assign-maintainer.yml index 80b2ec846..292bae4e7 100644 --- a/.github/workflows/auto-assign-maintainer.yml +++ b/.github/workflows/auto-assign-maintainer.yml @@ -14,7 +14,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.4 + - uses: kentaro-m/auto-assign-action@v1.2.5 with: configuration-path: .github/auto-assign-core.yml distribution: @@ -23,7 +23,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.4 + - uses: kentaro-m/auto-assign-action@v1.2.5 with: configuration-path: .github/auto-assign-distribution.yml implementation: @@ -32,7 +32,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.4 + - uses: kentaro-m/auto-assign-action@v1.2.5 with: configuration-path: .github/auto-assign-implementation.yml learning: @@ -41,7 +41,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.4 + - uses: kentaro-m/auto-assign-action@v1.2.5 with: configuration-path: .github/auto-assign-learning.yml platform: @@ -50,7 +50,7 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.4 + - uses: kentaro-m/auto-assign-action@v1.2.5 with: configuration-path: .github/auto-assign-platform.yml bat: @@ -59,6 +59,6 @@ jobs: runs-on: - ubuntu-latest steps: - - uses: kentaro-m/auto-assign-action@v1.2.4 + - uses: kentaro-m/auto-assign-action@v1.2.5 with: configuration-path: .github/auto-assign-bat.yml From d18479bbffe6e080cca90884e399f68be27fbd47 Mon Sep 17 00:00:00 2001 From: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> Date: Mon, 6 Mar 2023 14:39:52 -0700 Subject: [PATCH 319/372] Update 0096-remove-stacks-mixins.md Signed-off-by: Joe Kimmel <86852107+joe-kimmel-vmw@users.noreply.github.com> --- text/0096-remove-stacks-mixins.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/text/0096-remove-stacks-mixins.md b/text/0096-remove-stacks-mixins.md index 31dee5d68..34f17ea98 100644 --- a/text/0096-remove-stacks-mixins.md +++ b/text/0096-remove-stacks-mixins.md @@ -59,8 +59,8 @@ For Windows-based images, Distribution should be empty. Version should be the [s The `stacks` list in `buildpack.toml` is replaced by a `targets` list, where each entry corresponds to a different buildpack image that is exported into a [manifest index](https://github.com/opencontainers/image-spec/blob/master/image-index.md). Each entry may contain multiple valid values for Distribution and/or Version, but only a single OS, Architecture, and Variant. -If the `targets` list is empty and `/bin/build` is present, a target with `os = "linux"` and `arch = "x86_64"` is assumed by tools reading `buildpack.toml`. -If the `targets` list is empty and `/bin/build.bat` or `/bin/build.exe` is present, a target with `os = "windows"` and `arch = "x86_64"` is assumed by tools reading `buildpack.toml`. +If the `targets` list is empty and `/bin/build` is present, a target with `os = "linux"` and `arch = "amd64"` is assumed by tools reading `buildpack.toml`. +If the `targets` list is empty and `/bin/build.bat` or `/bin/build.exe` is present, a target with `os = "windows"` and `arch = "amd64"` is assumed by tools reading `buildpack.toml`. App image builds fail if the build image and selected run image have mismatched metadata. We may introduce flags or additional labels to skip this validation (e.g., for cross-compilation or minimal runtime base images). An image without a specified Distribution is compatible with images specifying any Distribution. @@ -73,14 +73,14 @@ When an app image is rebased, `rebaser` must fail if the new run image and previ ```toml [[targets]] os = "linux" -arch = "x86_64" +arch = "amd64" [[targets.distributions]] name = "ubuntu" versions = ["18.04", "20.04"] [[targets]] os = "linux" -arch = "x86_64" +arch = "amd64" [[targets.distributions]] name = "ubuntu" versions = ["14.04", "16.04"] @@ -152,7 +152,7 @@ If the newly-specified field values are missing, the lifecycle and pack may used ``` config.os = "linux" -config.architecture = "x86_64" +config.architecture = "amd64" io.buildpacks.distribution.name = "ubuntu" io.buildpacks.distribution.version = "18.04" ``` @@ -169,7 +169,7 @@ to ```toml [[targets]] os = "linux" -arch = "x86_64" +arch = "amd64" [[targets.distributions]] name = "ubuntu" versions = ["18.04"] @@ -201,3 +201,12 @@ versions = ["18.04"] [spec-changes]: #spec-changes This RFC requires extensive changes to all specifications. + +## Amended +### Summary + +rename x86_64 -> amd64 in keeping with all other usages of arch. descriptors. + +### Motivation + +This is how we do it everywhere else, this is the way. From 49f1700110e6c9e66bf87531ded7dd0fb2c49d04 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Tue, 7 Mar 2023 16:01:44 -0500 Subject: [PATCH 320/372] RFC 0119 [#203] Signed-off-by: Natalie Arellano --- text/{0000-export-to-oci.md => 0119-export-to-oci.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-export-to-oci.md => 0119-export-to-oci.md} (99%) diff --git a/text/0000-export-to-oci.md b/text/0119-export-to-oci.md similarity index 99% rename from text/0000-export-to-oci.md rename to text/0119-export-to-oci.md index d1b806f22..2dacbc55e 100644 --- a/text/0000-export-to-oci.md +++ b/text/0119-export-to-oci.md @@ -3,10 +3,10 @@ - Name: Export to OCI format - Start Date: 2022-02-22 - Author(s): Juan Bustamante (@jjbustamante) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#203](https://github.com/buildpacks/rfcs/pull/203) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From b01e57faa84914ca4285062f43ac52c947aee888 Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Wed, 8 Mar 2023 20:28:57 -0700 Subject: [PATCH 321/372] first draft of CVE Backporting rfc Signed-off-by: Joe Kimmel --- text/0000-cvebackports.md | 118 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 text/0000-cvebackports.md diff --git a/text/0000-cvebackports.md b/text/0000-cvebackports.md new file mode 100644 index 000000000..12e0ef921 --- /dev/null +++ b/text/0000-cvebackports.md @@ -0,0 +1,118 @@ +# Meta +[meta]: #meta +- Name: CVE discretional Patching +- Start Date: 2023-03-08 +- Author(s): joe-kimmel-vmw, natalieparellano +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary +This RFC describes how maintainers MAY issue patch releases in response to critical and high severity CVEs being detected in past or current releases of the lifecycle binary. + +# Definitions +[definitions]: #definitions +CVE - Literally expands to “Common Vulnerabilities and Exposures” but in general it refers to a security gap which could be exploited by a malicious attacker, which can be fixed by patching a single component. + +CVE Severity: CVEs are announced with a severity score. While this score can vary between vendors, it is made by considering many factors including how easy it is to take advantage of the exploit, what resources are exposed by the exploit, and whether there is already a known exploit in circulation. See https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator + +Critical and High are the two highest severity ratings available under the current severity rating system. + + +# Motivation +[motivation]: #motivation + +- Why should we do this? + +Patching CVEs is industry best practice. By providing patched updates of recent releases we enable CNB-using organizations with low risk tolerance or high upgrade friction to make a minimal change to their runtime infrastructure that still keeps secure against known vulnerabilities. +Following patch version line architecture also boosts user trust vs. a pledge to roll forward security fixes to higher version lines with the promise of no breaking changes, which would be looked upon skeptically by some enterprise users. Those same users who are skeptical of any & all promises that new versions won’t have any regressions are especially likely to be running older versions in the wild, thus providing patches to older versions ensures that they can continue to use CNB lifecycle without risking exposure to the riskiest CVEs. + +- What use cases does it support? + +This is especially important for users who do not use CNB as part of a SaaS product, i.e. for “on-prem” deployments. These on-prem users are pulling down the provided lifecycle images and running the binaries on their own infrastructure, thus increasing their potential risk exposure and liability. + +- What is the expected outcome? + +Maintainers MAY issue patch releases in response to critical and high CVEs. Most importantly, users MAY upgrade to consume these patch releases in a timely manner if they are not comfortable consuming the latest version. + +# What it is +[what-it-is]: #what-it-is + +It is risk mitigation for consumer and enterprise customers who want to apply critical patches without any other changes that would come with a minor version upgrade. + +# How it Works +[how-it-works]: #how-it-works + +Patch releases will be published at the discretion of the maintainers in response to Critical and High CVEs. + +The lifecycle will still offer the same strong backwards compatibility guarantees as ever. + +Existing process (patch most recent version N until it becomes N-1) bumps dependencies and the go version each month irrespective of CVEs being present. This proposal does not involve changing that process. + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + +Maintaining past releases does take time. However under this proposal that maintenance is optional and performed at the discretion of the maintainers. + +For a vague guess at the volume of this work: In the year from March 2022 March 2023, the grype scanner found 3 High and 0 Critical CVEs in the 0.13.5 lifecycle binary, so there would have been at most 3 additional patch releases of that line. +Similarly, grype scanner found 1 High and 0 Critical CVEs in the 0.14.3 lifecycle image (from October 2022), and 0 High or Critical CVEs in the 0.15.3 lifecycle image (from Jan 2023). + + + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + - Only Patch the Most Recent Release: This is a fine idea but it only works in a world where all users are willing to migrate to the latest release. This RFC addresses the wants and needs of users who are not comfortable performing minor version upgrades, even with backwards compatibility guarantees, in order to address CVEs. + - Only Patch development trunk, and wait for the next release: This also only works in a world where minor version upgrades are seen as cheap or low-risk to perform. + - Don’t Patch CVEs: we probably wouldn’t ever do this option. + + +- Why is this proposal the best? +This is the only proposal palatable to orgs that view minor releases as high-risk and/or expensive. +- What is the impact of not doing this? +We risk alienating some enterprise users of CNB. + +# Prior Art +[prior-art]: #prior-art + +N/A + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A beyond general feedback and consensus. + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +N/A + +# History +[history]: #history + + From 1019f08613d6b20afd2211438164ce47136f4e9b Mon Sep 17 00:00:00 2001 From: Joe Kimmel Date: Thu, 9 Mar 2023 09:42:20 -0700 Subject: [PATCH 322/372] one more note re. motivation Signed-off-by: Joe Kimmel --- text/0000-cvebackports.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-cvebackports.md b/text/0000-cvebackports.md index 12e0ef921..ce02872bf 100644 --- a/text/0000-cvebackports.md +++ b/text/0000-cvebackports.md @@ -34,6 +34,9 @@ Following patch version line architecture also boosts user trust vs. a pledge to This is especially important for users who do not use CNB as part of a SaaS product, i.e. for “on-prem” deployments. These on-prem users are pulling down the provided lifecycle images and running the binaries on their own infrastructure, thus increasing their potential risk exposure and liability. +Additionally, as we do have concrete plans to deprecate platform API versions in the lifecycle this calendar year, there's increased likelihood to learn of +other lifecycle consumers who are not ready to upgrade and who would appreciate patch releases. + - What is the expected outcome? Maintainers MAY issue patch releases in response to critical and high CVEs. Most importantly, users MAY upgrade to consume these patch releases in a timely manner if they are not comfortable consuming the latest version. From d807cc425f08df5d3feb64e566681d9ec3eaf91b Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 11 Apr 2023 12:39:50 -0500 Subject: [PATCH 323/372] Adding buildpack community organization criteria table Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 50 ++++++++++++++++++++++-------- 1 file changed, 37 insertions(+), 13 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 1f8d82462..657729844 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -11,7 +11,30 @@ # Summary -This RFC proposes the donation of the open-source project [kpack](https://github.com/pivotal/kpack/) into the [Cloud Native Buildpack](https://buildpacks.io/) (CNB) ecosystem. +This RFC proposes the donation of the open-source project [kpack](https://github.com/pivotal/kpack/) into the [Cloud Native Community Organization Buildpack](https://github.com/buildpacks-community). + +Following the process defined in the [Buildpack Commnity RFC](https://github.com/buildpacks/rfcs/blob/main/text/0117-buildpacks-community.md) the following table presents the criteria used to evaluate the project. + +| Criteria | Evidence | +|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| +| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | +| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | +| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | +| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149/files) | +| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | +| The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. || +| The list of project maintainers must be publicly available and controlled through a Github team. || +| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | +| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section | +| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). || +| There should have visible automated testing for all repositories that are part of the project. || +| The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || +| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149/files) | +| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. || +| The has a file - ADOPTERS.md: A list of adopters of the project. | See Adopters [Pull Request](https://github.com/pivotal/kpack/pull/1142) | +| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See Versioning [Pull Request](https://github.com/pivotal/kpack/pull/1150) | +| The has a file - RELEASE.md: A guide to how releases are done for the project. || +| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149/files) | # Definitions @@ -307,26 +330,27 @@ Based on the [CNB governance policy](https://github.com/buildpacks/community/blo How do migrate roles and responsibilities into the CNB governance process? -Currently the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team) already has a **team lead** assigned and, by definition, each team can have only one **team lead**. In order to provide the current [kpack](https://github.com/pivotal/kpack/) team with the same accountability for the migrated repositories the proposal is to follow the guidelines describe on the [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) +Currently, the [CNB Platform Team](https://github.com/buildpacks/community/blob/main/TEAMS.md#Platform-Team) already has a **team lead** assigned and, by definition, each team can have only one **team lead**. In order to provide the current [kpack](https://github.com/pivotal/kpack/) team with the same accountability for the migrated repositories the proposal is to follow the guidelines describe on the [Component Maintainer Role RFC](https://github.com/buildpacks/rfcs/pull/234) The [kpack's](https://github.com/pivotal/kpack/) maintainers that will be nominated as **component maintainer** in CNB are: -| Name | Github account | Organization | -| --- | --- | --- | -| Matthew McNew| [@matthewmcnew](https://github.com/matthewmcnew)| VMware| -| Tom Kennedy | [@tomkennedy513](https://github.com/tomkennedy513) | VMware | -| Yael Harel | [@yaelharel](https://github.com/yaelharel) | VMware| -| Daniel Chen |[@chenbh](https://github.com/chenbh) | VMware| -| Juan Bustamante |[@jjbustamante](https://github.com/jjbustamante) | VMware| +| Name | Github account | Organization | +|------------------|----------------------------------------------------|--------------| +| Matthew McNew | [@matthewmcnew](https://github.com/matthewmcnew) | VMware | +| Tom Kennedy | [@tomkennedy513](https://github.com/tomkennedy513) | VMware | +| Nicholas Carlson | [@ncarlson](https://github.com/ncarlson) | VMWare | +| Yael Harel | [@yaelharel](https://github.com/yaelharel) | VMware | +| Daniel Chen | [@chenbh](https://github.com/chenbh) | VMware | +| Juan Bustamante | [@jjbustamante](https://github.com/jjbustamante) | VMware | Also those members are willing to become more involved with CNB projects and become **Platform maintainers** in the near future. Outside VMware, the following contributors manifested their desired to become [kpack's](https://github.com/pivotal/kpack/) **component maintainer**. -| Name | Github account | Organization | -| --- | --- | --- | -| Sambhav Kothari| [@samj1912](https://github.com/samj1912) | Bloomberg | -| Aidan Delaney| [@AidanDelaney](https://github.com/AidanDelaney) | Bloomberg | +| Name | Github account | Organization | +|-----------------|--------------------------------------------------|--------------| +| Sambhav Kothari | [@samj1912](https://github.com/samj1912) | Bloomberg | +| Aidan Delaney | [@AidanDelaney](https://github.com/AidanDelaney) | Bloomberg | #### RFC process From 0a7b1d1a2f9ba32e408f2c640969fc18566f36b9 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 11 Apr 2023 12:42:00 -0500 Subject: [PATCH 324/372] removing adopters from RFC and pointing to the kpack repo Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 657729844..8a20ee3f4 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -264,17 +264,6 @@ pie showData "Others" : 37 ``` -# Adopters - -[VMware](https://www.vmware.com/) is the most obvious company using [kpack](https://github.com/pivotal/kpack/) as a core component in their [Tanzu](https://tanzu.vmware.com/tanzu) offer. [kpack](https://github.com/pivotal/kpack/) has been used in the enterprise for years in hundreds of costumers. - -Some others companies using [kpack](https://github.com/pivotal/kpack/) are: - -- [Bloomberg](http://techatbloomberg.com/opensource) -- [WPengine](https://wpengine.com/) -- [Terasky](https://www.terasky.com/) -- [SAP](https://www.sap.com/) - # Migration ### Repositories @@ -296,7 +285,8 @@ For each repository ### CI / CD Pipelines -[kpack's](https://github.com/pivotal/kpack/) CI/CD infrastructure currently runs on internal VMware infrastructure. [kpack's](https://github.com/pivotal/kpack/) CI/CD pipelines will need to be rebuilt on [CNB](https://buildpacks.io/) infrastructure. +[kpack's](https://github.com/pivotal/kpack/) CI/CD pipelines were rebuilt to use [github actions](https://github.com/pivotal/kpack/tree/main/.github). +In order for [kpack's](https://github.com/pivotal/kpack/) to run windows acceptance tests it requires a kubernetes cluster with windows nodes. The hardware requirements are specify in the following section. ##### Hardware requirements From 8cb9a0ced17270eca3616d1ea84600131525ed72 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 11 Apr 2023 14:50:54 -0500 Subject: [PATCH 325/372] updating the destination github repository to point to the buildpacks-community Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 8a20ee3f4..d9c0db67a 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -274,9 +274,9 @@ The following table shows the candidates repositories to be transferred | Origin Repo | Description | Owner | Destination Repo | Owner | | --- | --- | --- | --- | --- | -| [https://github.com/pivotal/kpack](https://github.com/pivotal/kpack) | kpack source code | Pivotal | [https://github.com/buildpacks/kpack](https://github.com/buildpacks/kpack) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | -| [https://github.com/vmware-tanzu/kpack-cli](https://github.com/vmware-tanzu/kpack-cli) | kpack CLI | VMware | [https://github.com/buildpacks/kpack-cli](https://github.com/buildpacks/kpack-cli) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | -| [https://github.com/vmware-tanzu/homebrew-kpack-cli](https://github.com/vmware-tanzu/homebrew-kpack-cli) | Homebrew tap for the kpack CLI | VMware | [https://github.com/buildpacks/homebrew-kpack-cli](https://github.com/buildpacks/homebrew-kpack-cli) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | +| [https://github.com/pivotal/kpack](https://github.com/pivotal/kpack) | kpack source code | Pivotal | [https://github.com/buildpacks-community/kpack](https://github.com/buildpacks/kpack) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | +| [https://github.com/vmware-tanzu/kpack-cli](https://github.com/vmware-tanzu/kpack-cli) | kpack CLI | VMware | [https://github.com/buildpacks-community/kpack-cli](https://github.com/buildpacks/kpack-cli) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | +| [https://github.com/vmware-tanzu/homebrew-kpack-cli](https://github.com/vmware-tanzu/homebrew-kpack-cli) | Homebrew tap for the kpack CLI | VMware | [https://github.com/buildpacks-community/homebrew-kpack-cli](https://github.com/buildpacks/homebrew-kpack-cli) | [CNB Technical Oversight Committee](https://github.com/buildpacks/community/blob/main/GOVERNANCE.md#technical-oversight-committee) | For each repository From 3bb9ae491604b3d03e14f0b53d597fa03b636a44 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 12 Apr 2023 10:28:13 -0500 Subject: [PATCH 326/372] adding link to the DEVELOPMENT.md pull request Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index d9c0db67a..1ea6e63a6 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -15,26 +15,26 @@ This RFC proposes the donation of the open-source project [kpack](https://github Following the process defined in the [Buildpack Commnity RFC](https://github.com/buildpacks/rfcs/blob/main/text/0117-buildpacks-community.md) the following table presents the criteria used to evaluate the project. -| Criteria | Evidence | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| -| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | -| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | -| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | -| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149/files) | -| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | +| Criteria | Evidence | +|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------| +| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | +| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | +| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | +| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | | The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. || | The list of project maintainers must be publicly available and controlled through a Github team. || -| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | -| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section | +| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | +| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section | | The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). || | There should have visible automated testing for all repositories that are part of the project. || | The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || -| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149/files) | -| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. || -| The has a file - ADOPTERS.md: A list of adopters of the project. | See Adopters [Pull Request](https://github.com/pivotal/kpack/pull/1142) | -| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See Versioning [Pull Request](https://github.com/pivotal/kpack/pull/1150) | +| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See Development [Pull Request](https://github.com/pivotal/kpack/pull/1185) | +| The has a file - ADOPTERS.md: A list of adopters of the project. | See Adopters [Pull Request](https://github.com/pivotal/kpack/pull/1142) | +| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See Versioning [Pull Request](https://github.com/pivotal/kpack/pull/1150) | | The has a file - RELEASE.md: A guide to how releases are done for the project. || -| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149/files) | +| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | # Definitions From e27efb917dd00045f8d83f92c4ba8c1773d5e49e Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 13 Apr 2023 10:43:08 -0400 Subject: [PATCH 327/372] RFC 0120 [#281] Signed-off-by: Natalie Arellano --- text/{0000-cvebackports.md => 0120-cvebackports.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-cvebackports.md => 0120-cvebackports.md} (97%) diff --git a/text/0000-cvebackports.md b/text/0120-cvebackports.md similarity index 97% rename from text/0000-cvebackports.md rename to text/0120-cvebackports.md index ce02872bf..1d20a99ff 100644 --- a/text/0000-cvebackports.md +++ b/text/0120-cvebackports.md @@ -3,10 +3,10 @@ - Name: CVE discretional Patching - Start Date: 2023-03-08 - Author(s): joe-kimmel-vmw, natalieparellano -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#281](https://github.com/buildpacks/rfcs/pull/281) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: N/A # Summary From a4188254cea29cff989ec8f7c5aee4c255ec970e Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 19 Apr 2023 12:13:05 -0500 Subject: [PATCH 328/372] WIP - initial version Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 148 +++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 text/0000-pack-manifest-list-commands.md diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md new file mode 100644 index 000000000..bfe9db6c4 --- /dev/null +++ b/text/0000-pack-manifest-list-commands.md @@ -0,0 +1,148 @@ +# Meta +[meta]: #meta +- Name: Manifest List Commands for Pack +- Start Date: 2023-04-19 +- Author(s): [Juan Bustamante](Juan Bustamante) +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +The problem for adding support for multi-arch buildpacks can be divided into two parts: +- Support buildpack authors to **migrate their existing buildpacks** to support multi-arch . +- Support buildpack authors to **create new buildpacks and builders** that handle multi-arch from the beginning. + +This RFC proposes to create a new set of CRUD commands in pack to handle manifest lists, which will be used to support the first part of the problem. + +# Definitions +[definitions]: #definitions + +- Image Manifest: The image manifest provides a configuration and set of layers for a single container image for a specific architecture and operating system. See [spec](https://github.com/opencontainers/image-spec/blob/main/manifest.md) +- Image Index: The image index is a higher-level manifest which points to specific image manifests, ideal for one or more platforms. See [spec](https://github.com/opencontainers/image-spec/blob/main/image-index.md) + +# Motivation +[motivation]: #motivation + +- Why should we do this? + +The uses of ARM architecture in the cloud and edge computing has been growing rapidly. The CNCF community has been also growing in the last years, and there is a need to support multi-arch for all the projects. The buildpacks community is not an exception, issues like: +- [It would be nice to support easily creating a manifest list packages and builders](https://github.com/buildpacks/pack/issues/1460) +- [Provide a way to specify desired platform when creating packages and builders](https://github.com/buildpacks/pack/issues/1459) +- [Multi arch image build support](https://github.com/buildpacks/pack/issues/1570) + +Or the conversations around this topic in our [slack channel](https://cloud-native.slack.com/archives/C032LNSMY0P), even the [talk at Kubecon NA 2022](https://www.youtube.com/watch?v=Sdr5axlOnDI&list=PLj6h78yzYM2O5aNpRM71NQyx3WUe1xpTn&index=76) demonstrate the interest from the community in this feature. + +- What use cases does it support? + +Currently, buildpack authors can build and package their buildpacks for different OS and Architectures, but when they distribute them the URI for a buildpack can’t disambiguate, they need to use different tags to differentiate between them. This makes harder for users to consume those Buildpacks. +The solution is to share a single URI for all the different OS and Architectures, and the way to do that is using a manifest list. + +Adding commands to support the operations to handle the manifest list will allow buildpack authors to migrate their existing buildpacks to support multi-arch, without afecting their current existing process. + +- What is the expected outcome? + +The expected outcome is to have a set of commands in pack to handle manifest lists, for example: +- A command to create a manifest list from a set of images +- A command to push the manifest list to a registry +- A command to update or delete a manifest list + +# What it is +[what-it-is]: #what-it-is + +The proposal is to add a new command `pack manifest` and different subcommands. +- `pack manifest create` will create a local manifest list for annotating and pushing to a registry +- `pack manifest annotate` will add additional information like os, arch or variant to an existing local manifest list +- `pack manifest add` will add an image to an existing manifest list +- `pack manifest remove` will remove an image from a manifest list +- `pack manifest push` will push a manifest list to a registry + +Our target user affected by the feature is: Buildpack Authors + + + +# How it Works +[how-it-works]: #how-it-works + +This is the technical portion of the RFC, where you explain the design in sufficient detail. + +The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. + +# Migration +[migration]: #migration + +This section should document breaks to public API and breaks in compatibility due to this RFC's proposed changes. In addition, it should document the proposed steps that one would need to take to work through these changes. Care should be give to include all applicable personas, such as platform developers, buildpack developers, buildpack users and consumers of buildpack images. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +We should decide to do not add this feature and users could use tools like `docker` or `podman` to create and handle their manifest list, however this is a poor user experience forcing users to use different tools to fulfill scenarios that are part of the business domain of pack. + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + +We also have in mind to improve existing commands like `pack builder create` and `pack buildpack package` to support the creation of a manifest list without the need of a new command. However, this approach could not be suitable for buildpack authors who are maintaining existing buildpacks and they will need to change their current process to generate multi-arch images. + +- Why is this proposal the best? + +Because we will provide the tool to our end users to solve their problems without the need of using other tools. + +- What is the impact of not doing this? + +The impact of not doing this is that users will need to use other tools to create and handle their manifest list, which is a poor user experience. + +# Prior Art +[prior-art]: #prior-art + +These features are inspired in similar commands in other tools like: +- [docker](https://docs.docker.com/engine/reference/commandline/manifest/) +- [podman](https://docs.podman.io/en/v3.2.0/markdown/podman-manifest-create.1.html) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes +Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here. +Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc. +This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here. + +# History +[history]: #history + + From 2878868f8df805256804bad43c3cc4ab1250fba0 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 20 Apr 2023 08:11:36 -0500 Subject: [PATCH 329/372] Update text/0000-pack-manifest-list-commands.md Co-authored-by: Aidan Delaney Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md index bfe9db6c4..2d96778cc 100644 --- a/text/0000-pack-manifest-list-commands.md +++ b/text/0000-pack-manifest-list-commands.md @@ -53,7 +53,7 @@ The expected outcome is to have a set of commands in pack to handle manifest lis # What it is [what-it-is]: #what-it-is -The proposal is to add a new command `pack manifest` and different subcommands. +The proposal is to add a new _experimental_ command `pack manifest` and different subcommands. The `pack manifest` commands will initially be gated behind `pack config experimental`. The `pack manifest` commands will move from experimental status to supported status when maintainers deem it appropriate. - `pack manifest create` will create a local manifest list for annotating and pushing to a registry - `pack manifest annotate` will add additional information like os, arch or variant to an existing local manifest list - `pack manifest add` will add an image to an existing manifest list From ae208e288bd3fffaa90ddc11638027c1193fa31a Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 20 Apr 2023 14:10:37 -0500 Subject: [PATCH 330/372] Adding what is this section Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 88 +++++++++++++++++++++++- 1 file changed, 87 insertions(+), 1 deletion(-) diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md index 2d96778cc..cb8507d66 100644 --- a/text/0000-pack-manifest-list-commands.md +++ b/text/0000-pack-manifest-list-commands.md @@ -60,7 +60,93 @@ The proposal is to add a new _experimental_ command `pack manifest` and differen - `pack manifest remove` will remove an image from a manifest list - `pack manifest push` will push a manifest list to a registry -Our target user affected by the feature is: Buildpack Authors +Our target user affected by the feature is: **Buildpack Authors**. Let's see some examples of how this feature will work. + +Currently, if we check [sample-packages](https://hub.docker.com/r/cnbs/sample-package/tags) at dockerhub we will notice that we have a composed buildpack called `hello-universe` and we offer two tags to support different architectures: +- `cnbs/sample-package:hello-universe` for linux and +- `cnbs/sample-package:hello-universe-windows`. + +Maybe our linux version should be called `cnbs/sample-package:hello-universe-linux` to keep the same naming convention, but we will keep it as it is for simplicity. If we want to distribute the `hello-universe` buildpack for any **architecture/os/variant** combination we need to use a tool outside the CNB ecosystem to create a manifest list. But, with our proposal experimental commands on pack we can do: + +```bash +$ pack manifest create cnbs/sample-package:hello-multiarch-universe \ + cnbs/sample-package:hello-universe \ + cnbs/sample-package:hello-universe-windows +``` + +By default the command will create a manifest list in the local daemon using the docker media types [Version 2 schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/) with a content similar to: + +```json +{ + "schemaVersion": 2, + "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", + "manifests": [ + { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 226083, + "digest": "sha256: 87a832fd6a8d6995d336c740eb6f3da015401a6e564fcbe95ee1bf37557a8225", + "platform": { + "os": "linux", + "architecture": "" + } + }, + { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 226083, + "digest": "sha256:670d62fbee841d256a706801a03be9c84d37fc2cd6ef7538a7af9985c3d2ed8b", + "platform": { + "os": "windows", + "architecture": "" + } + } + ] +} +``` +The idea to save the manifest list locally is to allow the user to update the manifest before pushing it to a registry, +in this case, we need to define the **architecture** field because it is empty in our examples. + +We can use the `pack manifest annotate` command to add the architecture information: + +```bash +$ pack manifest annotate --arch amd64 cnbs/sample-package:hello-multiarch-universe cnbs/sample-package:hello-universe +$ pack manifest annotate --arch amd64 cnbs/sample-package:hello-multiarch-universe cnbs/sample-package:hello-universe-windows +``` + +After executing these commands, our local manifest list will be updated as follows: + +```json +{ + "schemaVersion": 2, + "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", + "manifests": [ + { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 940, + "digest": "sha256:87a832fd6a8d6995d336c740eb6f3da015401a6e564fcbe95ee1bf37557a8225", + "platform": { + "architecture": "amd64", + "os": "linux" + } + }, + { + "mediaType": "application/vnd.docker.distribution.manifest.v2+json", + "size": 1148, + "digest": "sha256:670d62fbee841d256a706801a03be9c84d37fc2cd6ef7538a7af9985c3d2ed8b", + "platform": { + "architecture": "amd64", + "os": "windows" + } + } + ] +} +``` + +Finally, our manifest list is ready to be pushed to a registry, and we can use the `pack manifest push` command to do it: + +```bash +$ pack manifest push cnbs/sample-package:hello-multiarch-universe +``` +And our manifest list should be published at [dockerhub](https://hub.docker.com/r/cnbs/sample-package/tags) as `cnbs/sample-package:hello-multiarch-universe`, asuming that we have the proper credentials to push the image. - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) From 3086c836389231984046d47238d2ccc14ceb2b04 Mon Sep 17 00:00:00 2001 From: Ed Morley <501702+edmorley@users.noreply.github.com> Date: Fri, 28 Apr 2023 15:48:36 +0100 Subject: [PATCH 332/372] Fix example process definition in RFC 0093 RFC 0093 contains a number of examples of how to migrate a buildpack away from `direct=false` mode. One of these examples was missing the "bash" command prior to the "-c" argument. Signed-off-by: Ed Morley <501702+edmorley@users.noreply.github.com> --- text/0093-remove-shell-processes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0093-remove-shell-processes.md b/text/0093-remove-shell-processes.md index 269fee7c8..9a7f1876f 100644 --- a/text/0093-remove-shell-processes.md +++ b/text/0093-remove-shell-processes.md @@ -108,7 +108,7 @@ Using the new API this process could look like: ``` [[processes]] type = "bash" -command = ["-c", "dotnet", "my-app.dll", "--urls", "http://0.0.0.0:${PORT:-8080}"] +command = ["bash", "-c", "dotnet", "my-app.dll", "--urls", "http://0.0.0.0:${PORT:-8080}"] default = true ``` Things to note: @@ -400,4 +400,4 @@ In addition to the changes described originally in 0093 we'd like some way of ve Why was this amendment necessary? -The RFC text should reflect what was actually implemented / agreed upon to avoid confusion. \ No newline at end of file +The RFC text should reflect what was actually implemented / agreed upon to avoid confusion. From 8c8d7574ca324e73702de8132714f34c347de6f7 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 17 May 2023 17:40:59 -0500 Subject: [PATCH 333/372] updating latest changes for merged PR and removing Nicholas Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 33 +++++++++++++++--------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 1ea6e63a6..220f0de5b 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -15,26 +15,26 @@ This RFC proposes the donation of the open-source project [kpack](https://github Following the process defined in the [Buildpack Commnity RFC](https://github.com/buildpacks/rfcs/blob/main/text/0117-buildpacks-community.md) the following table presents the criteria used to evaluate the project. -| Criteria | Evidence | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------| -| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | -| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | -| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | -| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | -| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | +| Criteria | Evidence | +|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------| +| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | +| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | +| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | +| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | | The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. || | The list of project maintainers must be publicly available and controlled through a Github team. || -| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | -| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section | -| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). || -| There should have visible automated testing for all repositories that are part of the project. || +| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | +| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | +| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | +| There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | | The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || -| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | -| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See Development [Pull Request](https://github.com/pivotal/kpack/pull/1185) | -| The has a file - ADOPTERS.md: A list of adopters of the project. | See Adopters [Pull Request](https://github.com/pivotal/kpack/pull/1142) | -| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See Versioning [Pull Request](https://github.com/pivotal/kpack/pull/1150) | +| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | +| The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | +| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | | The has a file - RELEASE.md: A guide to how releases are done for the project. || -| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | # Definitions @@ -328,7 +328,6 @@ The [kpack's](https://github.com/pivotal/kpack/) maintainers that will be nomina |------------------|----------------------------------------------------|--------------| | Matthew McNew | [@matthewmcnew](https://github.com/matthewmcnew) | VMware | | Tom Kennedy | [@tomkennedy513](https://github.com/tomkennedy513) | VMware | -| Nicholas Carlson | [@ncarlson](https://github.com/ncarlson) | VMWare | | Yael Harel | [@yaelharel](https://github.com/yaelharel) | VMware | | Daniel Chen | [@chenbh](https://github.com/chenbh) | VMware | | Juan Bustamante | [@jjbustamante](https://github.com/jjbustamante) | VMware | From fa6df69fd813adbc671701e717e9ddd3b206d15d Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Thu, 18 May 2023 09:04:47 +0100 Subject: [PATCH 334/372] Update text/0000-kpack-donation-to-cnb.md Co-authored-by: Terence Lee Signed-off-by: Sambhav Kothari --- text/0000-kpack-donation-to-cnb.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 220f0de5b..23c1d2a9f 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -11,7 +11,7 @@ # Summary -This RFC proposes the donation of the open-source project [kpack](https://github.com/pivotal/kpack/) into the [Cloud Native Community Organization Buildpack](https://github.com/buildpacks-community). +This RFC proposes the donation of the open-source project [kpack](https://github.com/pivotal/kpack/) into the [Cloud Native Buildpacks Community Organization](https://github.com/buildpacks-community) as a vendor neutral staging ground under the CNB governance umbrella. Once the project is deemed sufficiently mature, the project will be moved under the [Cloud Native Buildpacks Organization](https://github.com/buildpacks). Following the process defined in the [Buildpack Commnity RFC](https://github.com/buildpacks/rfcs/blob/main/text/0117-buildpacks-community.md) the following table presents the criteria used to evaluate the project. From ef5f340da20cc7be31712ca58c7a6abfbd51ca5f Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 18 May 2023 14:38:23 -0500 Subject: [PATCH 335/372] Adding reference to Release.md PR Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 23c1d2a9f..68565716d 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -33,7 +33,7 @@ Following the process defined in the [Buildpack Commnity RFC](https://github.com | The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | | The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | | The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | -| The has a file - RELEASE.md: A guide to how releases are done for the project. || +| The has a file - RELEASE.md: A guide to how releases are done for the project. | See Release [Pull Request](https://github.com/pivotal/kpack/pull/1216) | | The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | # Definitions From 2f85dba546c463a04f9d57e9af76abc8cdd2b741 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 18 May 2023 15:35:52 -0500 Subject: [PATCH 336/372] adding go-licenses report link Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 35 +++++++++++++++--------------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 68565716d..4e2924627 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -15,26 +15,27 @@ This RFC proposes the donation of the open-source project [kpack](https://github Following the process defined in the [Buildpack Commnity RFC](https://github.com/buildpacks/rfcs/blob/main/text/0117-buildpacks-community.md) the following table presents the criteria used to evaluate the project. -| Criteria | Evidence | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------| -| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | -| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | -| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | -| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | -| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | +| Criteria | Evidence | +|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------| +| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | +| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | + | List of all external dependencies with licensing info and they’re permissively licensed with a Apache 2.0 compatible license | See [go-licenses](https://github.com/google/go-licenses) [report](https://drive.google.com/file/d/1SvPdl69Lhn0TTx_TaxcesfajuYY9uiNd/view?usp=share_link) | +| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | +| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | | The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. || | The list of project maintainers must be publicly available and controlled through a Github team. || -| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | -| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | -| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | -| There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | +| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | +| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | +| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | +| There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | | The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || -| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | -| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | -| The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | -| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | -| The has a file - RELEASE.md: A guide to how releases are done for the project. | See Release [Pull Request](https://github.com/pivotal/kpack/pull/1216) | -| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | +| The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | +| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | +| The has a file - RELEASE.md: A guide to how releases are done for the project. | See Release [Pull Request](https://github.com/pivotal/kpack/pull/1216) | +| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | # Definitions From b0a985efa084fb622db46cdbecbcdbe81f769d12 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 18 May 2023 15:37:48 -0500 Subject: [PATCH 337/372] adding go-licenses report link Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 36 +++++++++++++++--------------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 4e2924627..9d355d771 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -15,27 +15,27 @@ This RFC proposes the donation of the open-source project [kpack](https://github Following the process defined in the [Buildpack Commnity RFC](https://github.com/buildpacks/rfcs/blob/main/text/0117-buildpacks-community.md) the following table presents the criteria used to evaluate the project. -| Criteria | Evidence | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------| -| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | -| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | - | List of all external dependencies with licensing info and they’re permissively licensed with a Apache 2.0 compatible license | See [go-licenses](https://github.com/google/go-licenses) [report](https://drive.google.com/file/d/1SvPdl69Lhn0TTx_TaxcesfajuYY9uiNd/view?usp=share_link) | -| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | -| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | -| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | +| Criteria | Evidence | +|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | +| The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | + | List of all external dependencies with licensing info and they’re permissively licensed with a Apache 2.0 compatible license | See [report](https://drive.google.com/file/d/1SvPdl69Lhn0TTx_TaxcesfajuYY9uiNd/view?usp=share_link) generated using [go-licenses](https://github.com/google/go-licenses) | +| It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | +| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | | The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. || | The list of project maintainers must be publicly available and controlled through a Github team. || -| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | -| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | -| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | -| There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | +| The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | +| All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | +| The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | +| There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | | The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || -| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | -| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | -| The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | -| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | -| The has a file - RELEASE.md: A guide to how releases are done for the project. | See Release [Pull Request](https://github.com/pivotal/kpack/pull/1216) | -| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | +| The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | +| The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | +| The has a file - RELEASE.md: A guide to how releases are done for the project. | See Release [Pull Request](https://github.com/pivotal/kpack/pull/1216) | +| The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | # Definitions From 7840070df2a390b387cf2849cb09374f12fb7815 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 23 May 2023 15:50:53 -0500 Subject: [PATCH 338/372] updating some links to require documents Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 9d355d771..f85501fd3 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -23,14 +23,14 @@ Following the process defined in the [Buildpack Commnity RFC](https://github.com | It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | | The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | | The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | -| The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. || +| The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. | See Governance [Pull Request](https://github.com/pivotal/kpack/pull/1220) | | The list of project maintainers must be publicly available and controlled through a Github team. || | The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | | All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | | The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | | There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | | The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || -| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See Contributing [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See [Contributing](https://github.com/pivotal/kpack/blob/main/CONTRIBUTING.md) | | The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | | The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | | The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | From 30ce074e82fa2626135af7b496c0ab65092478c5 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Wed, 24 May 2023 09:29:50 -0500 Subject: [PATCH 339/372] updating link to RELEASE.md Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index f85501fd3..7e4ced6a5 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -34,7 +34,7 @@ Following the process defined in the [Buildpack Commnity RFC](https://github.com | The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | | The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | | The has a file - VERSIONING.md: A guide to how versioning is done for the project. | See [Versioning](https://github.com/pivotal/kpack/blob/main/VERSIONING.md) | -| The has a file - RELEASE.md: A guide to how releases are done for the project. | See Release [Pull Request](https://github.com/pivotal/kpack/pull/1216) | +| The has a file - RELEASE.md: A guide to how releases are done for the project. | See [Release](https://github.com/pivotal/kpack/blob/main/RELEASE.md) | | The has a file - SECURITY.md: A guide to how security vulnerabilities should be reported. | See Security [Pull Request](https://github.com/pivotal/kpack/pull/1149) | # Definitions @@ -329,11 +329,10 @@ The [kpack's](https://github.com/pivotal/kpack/) maintainers that will be nomina |------------------|----------------------------------------------------|--------------| | Matthew McNew | [@matthewmcnew](https://github.com/matthewmcnew) | VMware | | Tom Kennedy | [@tomkennedy513](https://github.com/tomkennedy513) | VMware | -| Yael Harel | [@yaelharel](https://github.com/yaelharel) | VMware | | Daniel Chen | [@chenbh](https://github.com/chenbh) | VMware | | Juan Bustamante | [@jjbustamante](https://github.com/jjbustamante) | VMware | -Also those members are willing to become more involved with CNB projects and become **Platform maintainers** in the near future. +Also, those members are willing to become more involved with CNB projects and become **Platform maintainers** in the near future. Outside VMware, the following contributors manifested their desired to become [kpack's](https://github.com/pivotal/kpack/) **component maintainer**. From 9ff4797e8091288afa916a7fee2eb1332d398819 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Fri, 9 Jun 2023 15:44:53 -0700 Subject: [PATCH 340/372] RFC for 2023H2 Roadmap Signed-off-by: Terence Lee --- text/0000-2023H2-roadmap.md | 171 ++++++++++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 text/0000-2023H2-roadmap.md diff --git a/text/0000-2023H2-roadmap.md b/text/0000-2023H2-roadmap.md new file mode 100644 index 000000000..5516fbe45 --- /dev/null +++ b/text/0000-2023H2-roadmap.md @@ -0,0 +1,171 @@ +# Meta +[meta]: #meta +- Name: 2023H2 Roadmap +- Start Date: 2023-06-07 +- Author(s): hone +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +This RFC details the second half of the 2023 Roadmap leading up to KubeCon NA. + +# Definitions +[definitions]: #definitions + +## Readmap Status Items +* Finished - The item has already been completed and there is no need to put it on the current roadmap. +* Continue - More work is needed and it will be continued as part of the current roadmap. +* Defer - This item will be parked for now and doesn't make the cut for the current roadmap. + +# Motivation +[motivation]: #motivation + +With KubeCon NA coming up around the corner, it's time to plan out what we want to achieve as a project. It's also an opportunity to review items from the H1 roadmap. + +# What it is +[what-it-is]: #what-it-is + +## 2023H2 Roadmap + +This roadmap is going to be split into two sections covering the first the H1 Roadmap and status, as well as new items we want to tackle leading up to KubeCon NA. + +### Items from [2023H1] +[items-from-2023h1]: #items-from-2023h1 + +As a project, we've made good progress on our H1 roadmap, but still need some more time to get some of them across the finish line. + +#### Release Base Image Extension +* Owner: @natalieparellano +* Status: Continue +* Links: [RFC](https://github.com/buildpacks/rfcs/blob/main/text/0105-dockerfiles.md) + +This is close and will be released as part of Platform `0.12`, Buildpack `0.10`, lifecycle `0.17.0`, and pack `0.30.0`. + +#### Remove Stacks & Mixins +* Owner: @jkutner +* Status: Continue +* Links: [RFC](https://github.com/buildpacks/rfcs/blob/main/text/0096-remove-stacks-mixins.md) + +This is close and will be released as part of Platform `0.12`, Buildpack `0.10`, lifecycle `0.17.0`, and pack `0.30.0`. + +#### Execution Environments RFC +* Owner: @hone +* Status: Continue +* Links: [RFC](https://github.com/buildpacks/rfcs/pull/274) + +The RFC is written, but feedback needs to be incoporated before re-opening for review. + +#### Project Health +* Owner: @samj1912 +* Status: Finished +* Links: [Buildpacks Community Organization RFC](https://github.com/buildpacks/rfcs/pull/273) + +The RFC has been merged and the buildpacks community GitHub org have been created. + +#### Pack Test Cleaning/Optimizations +* Owner: @dfreilich +* Status: Defer +* Links: [Pack Pull Request](https://github.com/buildpacks/pack/pull/1498) + +This item has been deferred for now, but we will work with anyone who wants to push this forward. + +### New Items + +#### Community Engagement Health Checks +* Owner: @microwavables (Team Lead sponsor @jkutner) +* Links: [VMware Tanzu Community Engagement Health Checks](https://github.com/vmware-tanzu/community-engagement/blob/main/HEALTHCHECKS.md) + +The project is lucky to have a Community Manager! This is one of the projects proposed by @microwavables to set a base line to measure how we're doing as a community. + +#### RFC for Buildpack Author Observability +* Owner: @joshwlewis (Team Lead sponsor @hone) +* Links: TBD + +Currently, Buildpack Authors have little to no tools around visibility with their buildpacks as they run on a platform, including `pack`. In some of the Heroku v2 buildpacks, they implemented logging that could be handed off to the platform by running `bin/report`. This work stream is about standardizing output for both successful AND failed builds that Buildpack Authors can use to instrument their buildpack. + +#### Private Registry Mirrors +* Owner: @jabrown85 +* Links: [RFC](https://github.com/buildpacks/rfcs/pull/285) + +A platform operator can configure registry mirrors that lifecycle could use without needing for the manifest to have to point to it. This will allow a platform to reduce the risk of service operations from external registry sources and reduce public network bandwidth usage. The resulting image when taken off platform will also function without needing access to the registry mirror. + +#### kpack Donation +* Owner: @jjbustamante (Team Lead sponsor @hone) +* Links: [RFC](https://github.com/buildpacks/rfcs/pull/235) + +`kpack` is being proposed to be donated as an open source project in the Cloud Native Buildpacks' new Community Organization as a vendor neutral staging ground. This will give the project space to grow the project contributor base from multiple vendors. While work has started on this in H1, this item represents our commitment as a project to see this through and set this project up for success under the CNB governance umbrella. + +#### Cosign Integration / OCI Image Manifest v1.1 +* Owner: @natalieparellano +* Links: [Cosign Integration RFC](https://github.com/buildpacks/rfcs/pull/195), [SBOM layer RFC](https://github.com/buildpacks/rfcs/pull/278), + +While CNBs support SBOMs today, they were designed a few years ago and tooling around them have been evolving. This work stream is about making CNBs integrate better with tools like [cosign's SBOM spec](https://github.com/sigstore/cosign/blob/main/specs/SBOM_SPEC.md) and the upcoming [OCI References](https://github.com/opencontainers/image-spec/issues/827) feature in OCI Image Manifest 1.1. + +#### Pack OCI Manifest Support +* Owner: @jjbustamante +* Links: [RFC](https://github.com/buildpacks/rfcs/pull/283) + +Multi-arch support has been a highly request feature with the growing popularity of the ARM architecture. In order to better support this with Buildpacks, the first step will be to able to use manifest lists to provide a single URI for Buildpacks that support multiple architectures. + +# How it Works +[how-it-works]: #how-it-works + +See [What it is](#what-it-is) for the details. We'll be following the same process from the [2023H1 Roadmap RFC](https://github.com/buildpacks/rfcs/blob/main/text/0118-2023H1-roadmap.md) if approved. + +# Migration +[migration]: #migration + +N/A + +# Drawbacks +[drawbacks]: #drawbacks + +- Agreeing to more work, while we still need to finish 2023H1 items. + +# Alternatives +[alternatives]: #alternatives + +- Do Nothing and just hunker down on our existing items. + +# Prior Art +[prior-art]: #prior-art + +See [Prior Art from 2023H1 Roadmap RFC](https://github.com/buildpacks/rfcs/blob/main/text/0118-2023H1-roadmap.md#prior-art). + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? +- What parts of the design do you expect to be resolved through implementation of the feature? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +N/A + +# History +[history]: #history + + From 3741035bae6feb255152e61ab2811d91f41cb667 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Thu, 22 Jun 2023 07:59:50 -0500 Subject: [PATCH 341/372] Updating the links to the governance.md file recently merged in kpack Signed-off-by: Juan Bustamante --- text/0000-kpack-donation-to-cnb.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0000-kpack-donation-to-cnb.md index 7e4ced6a5..b708bc876 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0000-kpack-donation-to-cnb.md @@ -19,17 +19,17 @@ Following the process defined in the [Buildpack Commnity RFC](https://github.com |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | The project must be a tooling, platform or integration that is related to Cloud Native Buildpacks. | See [Motivation](#motivation) section | | The project must be open source and licensed under Apache 2.0. | See [License](https://github.com/pivotal/kpack/blob/main/LICENSE) | - | List of all external dependencies with licensing info and they’re permissively licensed with a Apache 2.0 compatible license | See [report](https://drive.google.com/file/d/1SvPdl69Lhn0TTx_TaxcesfajuYY9uiNd/view?usp=share_link) generated using [go-licenses](https://github.com/google/go-licenses) | +| List of all external dependencies with licensing info and they’re permissively licensed with a Apache 2.0 compatible license | See [report](https://drive.google.com/file/d/1SvPdl69Lhn0TTx_TaxcesfajuYY9uiNd/view?usp=share_link) generated using [go-licenses](https://github.com/google/go-licenses) | | It must follow the Cloud Native Computing Foundation Code of Conduct. | See [Code of conduct](https://github.com/pivotal/kpack/blob/main/CODE_OF_CONDUCT.md) | -| The project must enable DCO signoff for all commits. | See Sign-off process [Pull Request](https://github.com/pivotal/kpack/pull/1149) | +| The project must enable DCO signoff for all commits. | See [Sign-off process](https://github.com/pivotal/kpack/blob/main/CONTRIBUTING.md#sign-off-process) | | The project must be open to contributions and have a public issue tracker. | See public [issue tracker](https://github.com/pivotal/kpack/issues) | -| The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. | See Governance [Pull Request](https://github.com/pivotal/kpack/pull/1220) | -| The list of project maintainers must be publicly available and controlled through a Github team. || +| The project must have a governance document that clearly defines the project maintainers and how they are elected. Each project may choose to define their own governance model as long as it is clearly documented and allows for project maintainers to be elected from the community. | See [Governance](https://github.com/pivotal/kpack/blob/main/GOVERNANCE.md) | +| The list of project maintainers must be publicly available and controlled through a Github team. | See [Maintainers](https://github.com/pivotal/kpack/blob/main/MAINTAINERS.md) | | The project must use a CODEOWNERS file to define the maintainers for each repository. The CODEOWNERS file should reference the Github team that controls the list of maintainers. | See [CODEOWNERS](https://github.com/pivotal/kpack/blob/main/CODEOWNERS) file | | All project contributors must be members of the Buildpacks community organization. | See [Team Roles](#team-roles) section and [People](https://github.com/orgs/buildpacks-community/people) in CNB community organization | | The project must be actively maintained (i.e. issues and pull requests must be addressed regularly, approved pull requests must be merged or updated in a timely manner, etc.). | See [issues](https://github.com/pivotal/kpack/issues) and [pull requests](https://github.com/pivotal/kpack/pulls) | | There should have visible automated testing for all repositories that are part of the project. | See [codecov](https://app.codecov.io/gh/pivotal/kpack) | -| The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. || +| The project maintainers must conform to a set of best effort SLOs around patching critical CVEs when applicable to the project. | | | The has a file - CONTRIBUTING.md: A guide to how contributors should submit patches and the expectations around code review. | See [Contributing](https://github.com/pivotal/kpack/blob/main/CONTRIBUTING.md) | | The has a file - DEVELOPMENT.md: A guide to how contributors should develop the project. | See [Development](https://github.com/pivotal/kpack/blob/main/DEVELOPMENT.md) | | The has a file - ADOPTERS.md: A list of adopters of the project. | See [Adopters](https://github.com/pivotal/kpack/blob/main/ADOPTERS.md) | From bfb4778d077e3bda49d4dd88bc7dc471e6b042c4 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Wed, 28 Jun 2023 11:31:04 -0500 Subject: [PATCH 342/372] Update 0115-rebase-immutable-image-ref.md Signed-off-by: Jesse Brown --- text/0115-rebase-immutable-image-ref.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0115-rebase-immutable-image-ref.md b/text/0115-rebase-immutable-image-ref.md index 02d548cb7..901911b49 100644 --- a/text/0115-rebase-immutable-image-ref.md +++ b/text/0115-rebase-immutable-image-ref.md @@ -3,9 +3,9 @@ - Name: Rebase by Image Digest Reference - Start Date: 2022-12-08 - Author(s): [@joeybrown-sf](https://github.com/joeybrown-sf) -- Status: Approved +- Status: Implemented - RFC Pull Request: [rfcs#262](https://github.com/buildpacks/rfcs/pull/262) -- CNB Pull Request: (leave blank) +- CNB Pull Request: https://github.com/buildpacks/lifecycle/pull/985 - CNB Issue: [buildpacks/lifecycle#983](https://github.com/buildpacks/lifecycle/issues/983) - Supersedes: N/A @@ -115,4 +115,4 @@ A brief description of the changes. ### Motivation Why was this amendment necessary? ----> \ No newline at end of file +---> From 6d37bbcde404abd83fcd89f9b91e747a90d6c76f Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Thu, 29 Jun 2023 08:53:27 -0700 Subject: [PATCH 343/372] add Export OCI Layout to the roadmap Signed-off-by: Terence Lee --- text/0000-2023H2-roadmap.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/text/0000-2023H2-roadmap.md b/text/0000-2023H2-roadmap.md index 5516fbe45..a3f87e190 100644 --- a/text/0000-2023H2-roadmap.md +++ b/text/0000-2023H2-roadmap.md @@ -107,11 +107,17 @@ A platform operator can configure registry mirrors that lifecycle could use with While CNBs support SBOMs today, they were designed a few years ago and tooling around them have been evolving. This work stream is about making CNBs integrate better with tools like [cosign's SBOM spec](https://github.com/sigstore/cosign/blob/main/specs/SBOM_SPEC.md) and the upcoming [OCI References](https://github.com/opencontainers/image-spec/issues/827) feature in OCI Image Manifest 1.1. #### Pack OCI Manifest Support -* Owner: @jjbustamante +* Owner: @jjbustamante (Team Lead sponsor @hone) * Links: [RFC](https://github.com/buildpacks/rfcs/pull/283) Multi-arch support has been a highly request feature with the growing popularity of the ARM architecture. In order to better support this with Buildpacks, the first step will be to able to use manifest lists to provide a single URI for Buildpacks that support multiple architectures. +#### Export to OCI Layout +* Owner: @jjbustamante (Team Lead sponsor @natalieparellano) +* Links: [RFC](https://github.com/buildpacks/rfcs/blob/main/text/0119-export-to-oci.md) + +The RFC has been merged and the implementation is expected to be released in experimental mode on pack *v0.30.0* + # How it Works [how-it-works]: #how-it-works From 174ce566b5ba7eeb2406351f5024962e01c07cd8 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 4 Jul 2023 15:51:12 -0500 Subject: [PATCH 344/372] adding details to the section how it works Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 47 +++++++++++++++++++++--- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md index b649e08d3..f8aa64a72 100644 --- a/text/0000-pack-manifest-list-commands.md +++ b/text/0000-pack-manifest-list-commands.md @@ -57,8 +57,10 @@ The proposal is to add a new _experimental_ command `pack manifest` and differen - `pack manifest create` will create a local manifest list for annotating and pushing to a registry - `pack manifest annotate` will add additional information like os, arch or variant to an existing local manifest list - `pack manifest add` will add an image to an existing manifest list -- `pack manifest remove` will remove an image from a manifest list +- `pack manifest delete` will delete a manifest list from local storage +- `pack manifest rm` will remove an image from a manifest list - `pack manifest push` will push a manifest list to a registry +- `pack manifest inspect` will show the manifest information stored in local storage Our target user affected by the feature is: **Buildpack Authors**. Let's see some examples of how this feature will work. @@ -66,7 +68,7 @@ Currently, if we check [sample-packages](https://hub.docker.com/r/cnbs/sample-pa - `cnbs/sample-package:hello-universe` for linux and - `cnbs/sample-package:hello-universe-windows`. -Maybe our linux version should be called `cnbs/sample-package:hello-universe-linux` to keep the same naming convention, but we will keep it as it is for simplicity. If we want to distribute the `hello-universe` buildpack for any **architecture/os/variant** combination we need to use a tool outside the CNB ecosystem to create a manifest list. But, with our proposal experimental commands on pack we can do: +Let's suppose our linux version is called `cnbs/sample-package:hello-universe-linux` to keep the same naming convention, but we will keep it as it is for simplicity. If we want to distribute the `hello-universe` buildpack for any **architecture/os/variant** combination we need to use a tool outside the CNB ecosystem to create a manifest list. With the proposed experimental commands on pack we can do: ```bash $ pack manifest create cnbs/sample-package:hello-multiarch-universe \ @@ -74,7 +76,7 @@ $ pack manifest create cnbs/sample-package:hello-multiarch-universe \ cnbs/sample-package:hello-universe-windows ``` -By default the command will create a manifest list in the local daemon using the docker media types [Version 2 schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/) with a content similar to: +By default, the command will create a manifest list in the local storage using the docker media types [Version 2 schema 2](https://docs.docker.com/registry/spec/manifest-v2-2/) with a content similar to: ```json { @@ -161,9 +163,44 @@ This provides a high level overview of the feature. # How it Works [how-it-works]: #how-it-works -This is the technical portion of the RFC, where you explain the design in sufficient detail. +The proposal is to implement an abstraction of an OCI *Image Index* and expose it to users through `pack manifest` commands. -The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. +## Image Index Abstraction + +```mermaid +classDiagram + ManifestList <|-- remote_ManifestList + ManifestList <|-- local_ManifestList + + class remote_ManifestList { + +NewManifestList(repoName string, keychain authn.Keychain) ManifestList + } + + class local_ManifestList { + +NewManifestList(repoName string, path string) ManifestList + } + + class ManifestList { + <> + +Add(repoName string) error + +Remove(repoName string) error + +Save() error + } +``` + +Two implementations: *remote* and *local* are propose, *remote* will take care of implementing the interaction with an OCI registry and *local* will deal with the local storage operations. + +### Component Diagram + +Using a [C4 component diagram](https://c4model.com/#ComponentDiagram), we can define the high level interaction on pack. This design follows the same pattern for each command already implemented. + +![](https://hackmd.io/_uploads/B1PpJ-fKh.png) + +- *Image Factory*: is responsible for instantiate the *Image Index* abstraction based on the configuration require, it could be a *remote* or a *local* implementation +- *Image Index*: is the abstraction defined which exposes the operation methods we want to offer to users + - As we can see, depending on the implementation it will interact with the file system or with a remote registry + +### Considerations # Migration [migration]: #migration From 261778bf271e228b906ec00eb8932c350e2dd1f7 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 4 Jul 2023 17:23:27 -0500 Subject: [PATCH 345/372] adding details to the section how it works Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 38 ++++++++++++++++++------ 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md index f8aa64a72..c1b709810 100644 --- a/text/0000-pack-manifest-list-commands.md +++ b/text/0000-pack-manifest-list-commands.md @@ -167,10 +167,19 @@ The proposal is to implement an abstraction of an OCI *Image Index* and expose i ## Image Index Abstraction +A new high level abstraction to represent an OCI Image Index is proposed, similar to the [Image](https://github.com/buildpacks/imgutil/blob/main/image.go) interface exposed in *imgutil* repository, +we proposed a new *ManifestList* interface to expose the behavior of an OCI Image Index. + ```mermaid classDiagram - ManifestList <|-- remote_ManifestList - ManifestList <|-- local_ManifestList + class ManifestList { + <> + +Add(repoName string) error + +Remove(repoName string) error + +Delete(additionalNames []string) error + +AnnotateManifest(manifestName string, opts AnnotateFields) error + +Save() error + } class remote_ManifestList { +NewManifestList(repoName string, keychain authn.Keychain) ManifestList @@ -179,16 +188,18 @@ classDiagram class local_ManifestList { +NewManifestList(repoName string, path string) ManifestList } - - class ManifestList { - <> - +Add(repoName string) error - +Remove(repoName string) error - +Save() error + + class AnnotateFields { + +String Architecture + +String OS + +String Variant } + + ManifestList <|-- remote_ManifestList + ManifestList <|-- local_ManifestList ``` -Two implementations: *remote* and *local* are propose, *remote* will take care of implementing the interaction with an OCI registry and *local* will deal with the local storage operations. +Two implementations: *remote* and *local* are proposed, *remote* will take care of implementing the interaction with an OCI registry and *local* will deal with the local storage operations. ### Component Diagram @@ -202,6 +213,15 @@ Using a [C4 component diagram](https://c4model.com/#ComponentDiagram), we can de ### Considerations +#### When a user wants to create a manifest list using a manifest outside the user's repo. + +Let's suppose the following user case: A user wants to create a manifest list `foo/my-manifest:my-tag` using a manifest outside his repository `foo` for example `other/external-manifest:latest`. + +In this case, pack will need to *copy* the external manifest `other/external-manifest:latest` into `foo` repository `foo/external-manifest:latest` and then uses this reference into the manifest list created. Pack should at least *warn* the user about this operation. + +#### When a user wants to create a manifest list referencing a manifest list + + # Migration [migration]: #migration From d5db1ec4abe1078ac074868cfcc7898faa44ed0f Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Thu, 6 Jul 2023 10:29:45 -0400 Subject: [PATCH 346/372] RFC 0121 [#235] Signed-off-by: Natalie Arellano --- ...ack-donation-to-cnb.md => 0121-kpack-donation-to-cnb.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-kpack-donation-to-cnb.md => 0121-kpack-donation-to-cnb.md} (99%) diff --git a/text/0000-kpack-donation-to-cnb.md b/text/0121-kpack-donation-to-cnb.md similarity index 99% rename from text/0000-kpack-donation-to-cnb.md rename to text/0121-kpack-donation-to-cnb.md index b708bc876..4316a97e6 100644 --- a/text/0000-kpack-donation-to-cnb.md +++ b/text/0121-kpack-donation-to-cnb.md @@ -3,10 +3,10 @@ - Name: kpack donation to CNB - Start Date: 2022-06-21 - Author(s): [Juan Bustamante](https://github.com/jjbustamante/) -- Status: Draft -- RFC Pull Request: (leave blank) +- Status: Approved +- RFC Pull Request: [rfcs#235](https://github.com/buildpacks/rfcs/pull/235) - CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) +- CNB Issue: N/A - Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) # Summary From 73126685161ea96924b269db3640701183a4ee3f Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Tue, 11 Jul 2023 18:21:54 -0400 Subject: [PATCH 347/372] adding another consideration Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md index c1b709810..f91e215f8 100644 --- a/text/0000-pack-manifest-list-commands.md +++ b/text/0000-pack-manifest-list-commands.md @@ -215,12 +215,22 @@ Using a [C4 component diagram](https://c4model.com/#ComponentDiagram), we can de #### When a user wants to create a manifest list using a manifest outside the user's repo. -Let's suppose the following user case: A user wants to create a manifest list `foo/my-manifest:my-tag` using a manifest outside his repository `foo` for example `other/external-manifest:latest`. +As a pack user I want to create a manifest list `foo/my-manifest:my-tag` using a manifest outside my repository `foo` for example `other/external-manifest:latest`. -In this case, pack will need to *copy* the external manifest `other/external-manifest:latest` into `foo` repository `foo/external-manifest:latest` and then uses this reference into the manifest list created. Pack should at least *warn* the user about this operation. +The user invokes a command similar to: `pack manifest create foo/my-manifest:my-tag other/external-manifest:latest` + +In this case, pack will need to *copy* the external image `other/external-manifest:latest` into `foo` repository `foo/external-manifest:latest` and then uses this reference to create the manifest list. In such as case `pack` should: + - *warn* the user about this operation, for example with a message + - *ask the user for confirmation* before executing the operation #### When a user wants to create a manifest list referencing a manifest list +As a pack user I want to create a manifest list using a reference to another manifest list + +The user invokes a command similar to: `pack manifest create foo/my-manifest:my-tag foo/another-manifest:latest` + +In this case, pack should: + - add into the **manifests** array of objects a reference to the manifest index using the media-type `application/vnd.oci.image.index.v1+json` (nested index) # Migration [migration]: #migration From 399bdc35631f0b09251a9115919ad0a19722ecc2 Mon Sep 17 00:00:00 2001 From: Sambhav Kothari Date: Sat, 15 Jul 2023 11:17:35 +0100 Subject: [PATCH 348/372] Update text/0000-2023H2-roadmap.md Signed-off-by: Sambhav Kothari --- text/0000-2023H2-roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-2023H2-roadmap.md b/text/0000-2023H2-roadmap.md index a3f87e190..9a795cad6 100644 --- a/text/0000-2023H2-roadmap.md +++ b/text/0000-2023H2-roadmap.md @@ -95,7 +95,7 @@ Currently, Buildpack Authors have little to no tools around visibility with thei A platform operator can configure registry mirrors that lifecycle could use without needing for the manifest to have to point to it. This will allow a platform to reduce the risk of service operations from external registry sources and reduce public network bandwidth usage. The resulting image when taken off platform will also function without needing access to the registry mirror. #### kpack Donation -* Owner: @jjbustamante (Team Lead sponsor @hone) +* Owner: @jjbustamante (Team Lead sponsor @samj1912) * Links: [RFC](https://github.com/buildpacks/rfcs/pull/235) `kpack` is being proposed to be donated as an open source project in the Cloud Native Buildpacks' new Community Organization as a vendor neutral staging ground. This will give the project space to grow the project contributor base from multiple vendors. While work has started on this in H1, this item represents our commitment as a project to see this through and set this project up for success under the CNB governance umbrella. From 68de434b3a222eb78fd9eeafec52d9536d096ac6 Mon Sep 17 00:00:00 2001 From: Juan Bustamante Date: Mon, 17 Jul 2023 12:26:15 -0400 Subject: [PATCH 349/372] adding details to the commands Signed-off-by: Juan Bustamante --- text/0000-pack-manifest-list-commands.md | 153 ++++++++++++++++++++++- 1 file changed, 151 insertions(+), 2 deletions(-) diff --git a/text/0000-pack-manifest-list-commands.md b/text/0000-pack-manifest-list-commands.md index f91e215f8..65fc80059 100644 --- a/text/0000-pack-manifest-list-commands.md +++ b/text/0000-pack-manifest-list-commands.md @@ -57,8 +57,8 @@ The proposal is to add a new _experimental_ command `pack manifest` and differen - `pack manifest create` will create a local manifest list for annotating and pushing to a registry - `pack manifest annotate` will add additional information like os, arch or variant to an existing local manifest list - `pack manifest add` will add an image to an existing manifest list -- `pack manifest delete` will delete a manifest list from local storage -- `pack manifest rm` will remove an image from a manifest list +- `pack manifest remove` will delete a manifest list from local storage +- `pack manifest rm` will remove an image from a manifest list in the local storage - `pack manifest push` will push a manifest list to a registry - `pack manifest inspect` will show the manifest information stored in local storage @@ -232,6 +232,155 @@ The user invokes a command similar to: `pack manifest create foo/my-manifest:my- In this case, pack should: - add into the **manifests** array of objects a reference to the manifest index using the media-type `application/vnd.oci.image.index.v1+json` (nested index) +### Commands details + +#### Create a Manifest List + +Pack will create a manifest a local manifest, it should handle the following scenarios: +- IF user references a manifest list that already exists in a registry: In this case, pack will save a local copy of the remote manifest list, this is useful for updating (adding, updating or deleting) images +- IF user references a manifest list that doesn't exist in a registry: pack will create a local representation of the manifest list that will only save on the remote registry if the user publish it + +```bash +manifest create generates a manifest list for a multi-arch image + +Usage: + pack manifest create [ ... ] [flags] + +Examples: +pack manifest create cnbs/sample-package:hello-multiarch-universe \ + cnbs/sample-package:hello-universe \ + cnbs/sample-package:hello-universe-windows + +Flags: + -f, --format string Format to save image index as ("OCI" or "V2S2") (default "v2s2") + --insecure Allow publishing to insecure registry + --publish Publish to registry + -r, --registry string Registry URL to publish the image index +``` + +#### Annotate (os/arch) a Manifest List + +Sometimes a manifest list could reference an image that doesn't specify the *architecture*, for example, [check](https://hub.docker.com/r/cnbs/sample-package/tags) our sample buildpack +packages. The `annotate` command allows users to update those values before pushing the manifest list a registry + +```bash +manifest annotate modifies a manifest list (Image index) and update the platform information for an image included in the manifest list. + +Usage: + pack manifest annotate [OPTIONS] [flags] + +Examples: +pack manifest annotate cnbs/sample-package:hello-universe-multiarch \ cnbs/sample-package:hello-universe --arch amd64 + +Flags: + --arch string Set the architecture + --os string Set the operating system + --variant string Set the architecture variant +``` + +#### Add an image to a Manifest List + +When a manifest list exits locally, user can add a new image to the manifest list using this command + +```bash +manifest add modifies a manifest list (Image index) and add a new image to the list of manifests. + +Usage: + pack manifest add [OPTIONS] [flags] + +Examples: +pack manifest add cnbs/sample-package:hello-multiarch-universe \ + cnbs/sample-package:hello-universe-riscv-linux + +Flags: + --all add all of the contents to the local list (applies only if is an index) + --arch string Set the architecture + --os string Set the operating system + --variant string Set the architecture variant +``` + +#### Remove an image from a Manifest List + +In the opposite case, users can remove existing images from a manifest list + +```bash +Delete one or more manifest lists from local storage + +Usage: + pack manifest remove [manifest-list] [manifest-list...] [flags] + +Examples: +pack manifest delete cnbs/sample-package:hello-multiarch-universe + +Flags: + -h, --help Help for 'remove' +``` + +#### Remove a local Manifest List + +Sometimes users can just experiment with the feature locally and they want to discard all the local information created by pack. `rm` command just delete the *local* manifest list + +```bash +manifest remove will remove the specified image manifest if it is already referenced in the index + +Usage: + pack manifest rm [manifest-list] [manifest] [flags] + +Examples: +pack manifest rm cnbs/sample-package:hello-multiarch-universe \ + cnbs/sample-package:hello-universe-windows + +Flags: + -h, --help Help for 'rm' + +``` + +#### Push a Manifest List to a remote registry + +Once a manifest list is ready to be publishe into the registry, the `push` command can be used + +```bash +manifest push pushes a manifest list (Image index) to a registry. + +Usage: + pack manifest push [OPTIONS] [flags] + +Examples: +pack manifest push cnbs/sample-package:hello-multiarch-universe + +Flags: + -f, --format string Manifest list type (oci or v2s2) to use when pushing the list (default is v2s2) + --insecure Allow publishing to insecure registry + -p, --purge Delete the manifest list or image index from local storage if pushing succeeds +``` + +#### Inspect a Manifest List + +Finally, the `inspect` command will help users to view how their local manifest list looks like + +```bash +manifest inspect shows the manifest information stored in local storage + +Usage: + pack manifest inspect [flags] + +Examples: +pack manifest inspect cnbs/sample-builder:multiarch + +Flags: + -h, --help Help for 'inspect' + +``` + +One important concern for users is to inspect the content of a multi-arch builder or buildpack if they are accessible behind a manifest list. The proposal is to implement a `platform` flag for commands: + +- `pack builder inspect` +- `pack buildpack inspect` + +The `--platform` flag specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). By default it will reference the host values. +The output of the commands should remain the same. + + # Migration [migration]: #migration From ef2ec1acf027db819a990713efff8738882512d4 Mon Sep 17 00:00:00 2001 From: Terence Lee Date: Wed, 19 Jul 2023 14:59:45 -0700 Subject: [PATCH 350/372] RFC 0122 --- .../{0000-2023H2-roadmap.md => 0122-2023H2-roadmap.md} | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) rename text/{0000-2023H2-roadmap.md => 0122-2023H2-roadmap.md} (96%) diff --git a/text/0000-2023H2-roadmap.md b/text/0122-2023H2-roadmap.md similarity index 96% rename from text/0000-2023H2-roadmap.md rename to text/0122-2023H2-roadmap.md index 9a795cad6..972ff9eef 100644 --- a/text/0000-2023H2-roadmap.md +++ b/text/0122-2023H2-roadmap.md @@ -3,11 +3,11 @@ - Name: 2023H2 Roadmap - Start Date: 2023-06-07 - Author(s): hone -- Status: Draft -- RFC Pull Request: (leave blank) -- CNB Pull Request: (leave blank) -- CNB Issue: (leave blank) -- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) +- Status: Approved +- RFC Pull Request: [rfcs#286](https://github.com/buildpacks/rfcs/pull/286) +- CNB Pull Request: +- CNB Issue: N/A +- Supersedes: [RFC 0118](https://github.com/buildpacks/rfcs/blob/main/text/0118-2023H1-roadmap.md) # Summary [summary]: #summary From fc19b12708556d5c414169845eaec2ba642860a5 Mon Sep 17 00:00:00 2001 From: Domenico Luciani Date: Thu, 20 Jul 2023 15:45:13 +0200 Subject: [PATCH 351/372] RFC Flatten builders and buildpacks Signed-off-by: Domenico Luciani --- text/XXX-flatten-feature.md | 325 ++++++++++++++++++++++++++++++++++++ 1 file changed, 325 insertions(+) create mode 100644 text/XXX-flatten-feature.md diff --git a/text/XXX-flatten-feature.md b/text/XXX-flatten-feature.md new file mode 100644 index 000000000..779be67bd --- /dev/null +++ b/text/XXX-flatten-feature.md @@ -0,0 +1,325 @@ +# Meta +[meta]: #meta +- Name: Flatten builders and buildpacks +- Start Date: 2023-07-13 +- Author(s): @jjbustamante, @dlion +- Status: Draft +- RFC Pull Request: (leave blank) +- CNB Pull Request: (leave blank) +- CNB Issue: (leave blank) +- Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC) + +# Summary +[summary]: #summary + +We propose to add new capabilities to the Pack tool that allow end users to reduce the number of Buildpack's layers in an OCI image by flattening according to their requirements. + +# Definitions +[definitions]: #definitions + +- Buildpack: A buildpack is a set of executables that inspects your app source code and creates a plan to build and run your application. +- Builder: A builder is an image that contains all the components necessary to execute a build. A builder image is created by taking a build image and adding a lifecycle, buildpacks, and files that configure aspects of the build including the buildpack detection order and the location(s) of the run image + +# Motivation +[motivation]: #motivation + +- Why should we do this? + +There is a limit in the number of layer an image can have, at least on Docker, which is *127*, this feature has being request by the community, issue [#1595](https://github.com/buildpacks/pack/issues/1595), as a workaround to solve error thrown by docker when the limit is reached + +- What use cases does it support? + +Buildpacks provider like Paketo have composite buildpacks with several layers, when they pull many of those together into a builder, hitting the layer limit for a container image happens very often. A feature for the Buildpack author to group the buildpacks by any attribute will allow them to squash those groups into one layer and reduce their total number of layers, avoiding the layer limit. + +- What is the expected outcome? + +When Buildpacks Authors execute commands like: + +`pack builder create ... ` or +`pack buildpack package ... ` + +The final OCI image artifact (A) SHOULD contain layers blobs with more than *one* buildpack according to the configuration provided by the user. If we compare an artifact (B) created *without* `` then: + +$numberOfBuildpackLayers(A) \leq numberOfBuildpackLayers(B)$ + +A and B MUST be otherwise interchangeable, only differing by their number of layers. + + +# What it is +[what-it-is]: #what-it-is + +The proposal is to include new experimental flags to the following commands on pack: + +- `pack builder create` +- `pack buildpack package` + +The new flags will move from experimental status to supported status when maintainers deem it appropriate. + +The new flags to be included are: + +- `--flatten` will flatten all the Buildpacks in one layer +- `--flatten --depth=` If there are composite buildpack to be flatten, `depth` is the high in the buildpack dependency tree to start flattening the buildpacks +- `--flatten --flatten-exclude=` will flatten all the Buildpacks in one layer excepts the ones specified under the `flatten-exclude` flag +- `--flatten --flatten-include=` will flatten the Buildpacks specified after the `flatten-include` flag into a single layer. Can be used more than once, with each use resulting in a single layer. + +We also need to define how a platform implementor needs to consume a flatten buildpackpage or builder. + +- When a platform consumes an OCI image artifact, they will need to inspect each buildpack layer blob and determine if the blob contains more than one buildpack, in such as case, they will need to process those buildpacks correctly. + + +# How it Works +[how-it-works]: #how-it-works + +Let's say we have a composite buildpack (CB1) with the following dependency tree: +```mermaid +flowchart TD + A[CB1] + A --> B[G1] + A --> C[G2] + B --> BPA[BP1] + B --> BPFOO[BP2] + B --> BPC[BP4] + C --> BPD[BP1] + C --> BPBAR[BP3] + C --> BPE[BP4] +``` + +Until now, when a buildpack like this is being shipped into an OCI image every individual buildpack is being saved in one layer, as a result we will have: + +$$ + layer_1 = [CB_1] \\ + layer_2 = [G_1] \\ + layer_3 = [BP_1] \\ + layer_4 = [BP_2] \\ + layer_5 = [BP_4] \\ + layer_6 = [G_2] \\ + layer_7 = [BP_3] \\ + total = \text{7 layers} +$$ + +Noticed that duplicated buildpacks are cleaned up. + +We can use the new `flatten` flag to reduce the number of OCI image layers used by the buildpacks in different ways. + +* `--flatten`: +Will flatten all buildpacks into one layer, the result will be: + +```mermaid +classDiagram + class Layer1 { + CB1 + G1 + BP1 + BP2 + BP4 + G2 + BP3 + } +``` + + +$$ + total = \text{1 layer} +$$ + +--- +* `--flatten --flatten-exclude=` i.e. `--flatten-exclude=` +Will flatten all buildpacks in 1 layer except for`BP2`, the result will be + +```mermaid +classDiagram + class Layer1 { + CB1 + G1 + BP1 + BP4 + G2 + BP3 + } + class Layer2 { + BP2 + } +``` + +$$ + total = \text{2 layers} +$$ + +--- + +* `--flatten --depth=0` +Will flatten all layers at the root (0) level, which is similar to flatten all, the result will be: + +```mermaid +classDiagram + class Layer1 { + CB1 + G1 + BP1 + BP2 + BP4 + G2 + BP3 + } +``` + +$$ + total = \text{1 layer} +$$ + +--- +Increasing the depth number will allow to flatten the tree deeper, +for example setting `--depth=1`, the result will be: + +```mermaid +classDiagram + class Layer1 { + CB1 + } + class Layer2 { + G1 + BP1 + BP2 + BP4 + } + class Layer3 { + G2 + BP3 + } +``` + + +$$ + total = \text{3 layers} +$$ + + +--- + +* `--flatten --flatten-include=` i.e. `--flatten --flatten-include= --flatten-include=` + +Will group the given buildpacks into one layer and keep the other ones as single layers buildpacks, the result will be: + +```mermaid +classDiagram + class Layer1 { + CB1 + } + class Layer2 { + G1 + } + class Layer3 { + BP1 + BP2 + } + class Layer4 { + G2 + } + class Layer5 { + BP3 + BP4 + } +``` + + +$$ + total = \text{5 layers} +$$ + + +# Migration +[migration]: #migration + +The current [distribution spec](https://github.com/buildpacks/spec/blob/main/distribution.md#buildpackage) defines: + +``` +Each buildpack layer blob MUST contain a single buildpack at the following file path: + +/cnb/buildpacks/// +``` + +A Buildpackage flattened with this new feature would not be consumable by older platform implementations because they are not expecting to find more than one buildpack on a blob layer. + + + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +Distributing a buildpackage with more than one buildpack in a layer blob adds complexity to platform implementors because they will need to do the opposite process when consuming the OCI images. + +It will add complexity to Buildpack Authors processes because they will need to think how to group their buildpacks together in a layer blob in an efficient way avoiding for example network issues when pulling new blobs from a registry. Size, frequency of change and other variables must be taken into consideration when buildpacks are flatten. + +It will create artifacts that are not consumable by older platforms. + + +# Alternatives +[alternatives]: #alternatives + +- What other designs have been considered? + +Some other alternatives mentioned are: squashing by the buildpack size or squashing a CNB Builder when the number of layers is reaching the limit, but those ideas, do not deal with the main problems of distributing more than one buildpack in a layer blob. + + +- Why is this proposal the best? + +Not sure if it is the best, but way to solve the `layer limit error` is to optimize the uses of the layer in a OCI image. + +- What is the impact of not doing this? + +Buildpack Authors and Platform Operators will keep seeing the layer limit error. + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to be resolved before this gets merged? + +We think the `--depth` command MAY not be helpful, after implementing a PoC. Should we get rid of this flag? + + + + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +Maybe, but it's not clear about this. + + + +# History +[history]: #history + +% \ No newline at end of file From c33e34be8340f4715c8dc1c3c8f4970eb5f585a2 Mon Sep 17 00:00:00 2001 From: Domenico Luciani Date: Tue, 1 Aug 2023 15:41:47 +0200 Subject: [PATCH 352/372] Renamed rfc Signed-off-by: Domenico Luciani --- text/{XXX-flatten-feature.md => 0000-flatten-feature.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename text/{XXX-flatten-feature.md => 0000-flatten-feature.md} (100%) diff --git a/text/XXX-flatten-feature.md b/text/0000-flatten-feature.md similarity index 100% rename from text/XXX-flatten-feature.md rename to text/0000-flatten-feature.md From 61e166c3b7111acffb87f9ad8684a0f0f79ff14c Mon Sep 17 00:00:00 2001 From: Domenico Luciani Date: Tue, 1 Aug 2023 17:11:53 +0200 Subject: [PATCH 353/372] Updated the RFC based on the feedback we got from our stakelholders Signed-off-by: Domenico Luciani --- text/0000-flatten-feature.md | 143 +++++------------------------------ 1 file changed, 20 insertions(+), 123 deletions(-) diff --git a/text/0000-flatten-feature.md b/text/0000-flatten-feature.md index 779be67bd..19924580d 100644 --- a/text/0000-flatten-feature.md +++ b/text/0000-flatten-feature.md @@ -38,7 +38,7 @@ When Buildpacks Authors execute commands like: `pack builder create ... ` or `pack buildpack package ... ` -The final OCI image artifact (A) SHOULD contain layers blobs with more than *one* buildpack according to the configuration provided by the user. If we compare an artifact (B) created *without* `` then: +The final OCI image artifact (A) SHOULD contain layers blobs with more than *one* buildpack according to the configuration provided by the user. If we compare an artifact (B) created *without* `` then: $numberOfBuildpackLayers(A) \leq numberOfBuildpackLayers(B)$ @@ -57,10 +57,7 @@ The new flags will move from experimental status to supported status when mainta The new flags to be included are: -- `--flatten` will flatten all the Buildpacks in one layer -- `--flatten --depth=` If there are composite buildpack to be flatten, `depth` is the high in the buildpack dependency tree to start flattening the buildpacks -- `--flatten --flatten-exclude=` will flatten all the Buildpacks in one layer excepts the ones specified under the `flatten-exclude` flag -- `--flatten --flatten-include=` will flatten the Buildpacks specified after the `flatten-include` flag into a single layer. Can be used more than once, with each use resulting in a single layer. +- `--flatten=` will flatten the Buildpacks specified after the `flatten` flag into a single layer. Can be used more than once, with each use resulting in a single layer. We also need to define how a platform implementor needs to consume a flatten buildpackpage or builder. @@ -86,119 +83,23 @@ flowchart TD Until now, when a buildpack like this is being shipped into an OCI image every individual buildpack is being saved in one layer, as a result we will have: -$$ - layer_1 = [CB_1] \\ - layer_2 = [G_1] \\ - layer_3 = [BP_1] \\ - layer_4 = [BP_2] \\ - layer_5 = [BP_4] \\ - layer_6 = [G_2] \\ - layer_7 = [BP_3] \\ - total = \text{7 layers} $$ - -Noticed that duplicated buildpacks are cleaned up. - -We can use the new `flatten` flag to reduce the number of OCI image layers used by the buildpacks in different ways. - -* `--flatten`: -Will flatten all buildpacks into one layer, the result will be: - -```mermaid -classDiagram - class Layer1 { - CB1 - G1 - BP1 - BP2 - BP4 - G2 - BP3 - } -``` - - -$$ - total = \text{1 layer} -$$ - ---- -* `--flatten --flatten-exclude=` i.e. `--flatten-exclude=` -Will flatten all buildpacks in 1 layer except for`BP2`, the result will be - -```mermaid -classDiagram - class Layer1 { - CB1 - G1 - BP1 - BP4 - G2 - BP3 - } - class Layer2 { - BP2 - } -``` - -$$ - total = \text{2 layers} -$$ - ---- - -* `--flatten --depth=0` -Will flatten all layers at the root (0) level, which is similar to flatten all, the result will be: - -```mermaid -classDiagram - class Layer1 { - CB1 - G1 - BP1 - BP2 - BP4 - G2 - BP3 - } -``` - -$$ - total = \text{1 layer} +layer_1 = [CB_1] \\ +layer_2 = [G_1] \\ +layer_3 = [BP_1] \\ +layer_4 = [BP_2] \\ +layer_5 = [BP_4] \\ +layer_6 = [G_2] \\ +layer_7 = [BP_3] \\ +total = \text{7 layers} $$ ---- -Increasing the depth number will allow to flatten the tree deeper, -for example setting `--depth=1`, the result will be: +Noticed that duplicated buildpacks are cleaned up. -```mermaid -classDiagram - class Layer1 { - CB1 - } - class Layer2 { - G1 - BP1 - BP2 - BP4 - } - class Layer3 { - G2 - BP3 - } -``` - - -$$ - total = \text{3 layers} -$$ - - ---- - -* `--flatten --flatten-include=` i.e. `--flatten --flatten-include= --flatten-include=` +We can use the new `flatten` flag to reduce the number of OCI image layers used by the buildpacks in different ways. -Will group the given buildpacks into one layer and keep the other ones as single layers buildpacks, the result will be: +* `--flatten=` i.e. `--flatten= --flatten=`: + Will group the given buildpacks into one layer and keep the other ones as single layers buildpacks, the result will be: ```mermaid classDiagram @@ -222,9 +123,11 @@ classDiagram ``` -$$ - total = \text{5 layers} $$ +total = \text{5 layers} +$$ + +--- # Migration @@ -270,20 +173,14 @@ Not sure if it is the best, but way to solve the `layer limit error` is to optim - What is the impact of not doing this? -Buildpack Authors and Platform Operators will keep seeing the layer limit error. +Buildpack Authors and Platform Operators will keep seeing the layer limit error. # Prior Art [prior-art]: #prior-art Discuss prior art, both the good and bad. -# Unresolved Questions -[unresolved-questions]: #unresolved-questions - -- What parts of the design do you expect to be resolved before this gets merged? - -We think the `--depth` command MAY not be helpful, after implementing a PoC. Should we get rid of this flag? - +--- \ No newline at end of file From 0728af7829c09c632dc3fd8329bdca8029614b03 Mon Sep 17 00:00:00 2001 From: Domenico Luciani Date: Mon, 4 Sep 2023 10:34:40 +0200 Subject: [PATCH 356/372] Cleaned the rfc removing anything not related to a Builder Signed-off-by: Domenico Luciani --- text/0000-flatten-feature.md | 55 +++++++++++++++++------------------- 1 file changed, 26 insertions(+), 29 deletions(-) diff --git a/text/0000-flatten-feature.md b/text/0000-flatten-feature.md index 3e5994235..bf3339d3c 100644 --- a/text/0000-flatten-feature.md +++ b/text/0000-flatten-feature.md @@ -1,6 +1,6 @@ # Meta [meta]: #meta -- Name: Flatten builders and buildpacks +- Name: Flatten builders - Start Date: 2023-07-13 - Author(s): @jjbustamante, @dlion - Status: Draft @@ -12,7 +12,9 @@ # Summary [summary]: #summary -We propose to add new capabilities to the Pack tool that allow end users to reduce the number of Buildpack's layers in an OCI image by flattening according to their requirements. +We propose to add new capabilities to the Pack tool that allow end-users to reduce the number of Buildpack's layers in a Builder by flattening some Buildpacks according to their requirements. + +This RFC mainly focus on applying this strategy to Builders only. # Definitions [definitions]: #definitions @@ -21,7 +23,7 @@ We propose to add new capabilities to the Pack tool that allow end users to redu - Builder: A builder is an image that contains all the components necessary to execute a build. A builder image is created by taking a build image and adding a lifecycle, buildpacks, and files that configure aspects of the build including the buildpack detection order and the location(s) of the run image - Component Buildpack: A component buildpack is a buildpack containing `/bin/detect` and `/bin/build` executables. Component buildpacks implement the [Buildpack Interface](https://github.com/buildpacks/spec/blob/main/buildpack.md). - Composite Buildpack: A composite buildpack is a buildpack containing an order definition in `buildpack.toml`. Composite buildpacks do not contain `/bin/detect` or `/bin/build` executables. They MUST be [resolvable](https://github.com/buildpacks/spec/blob/main/buildpack.md#order-resolution) into a collection of component buildpacks. -- Buildpackage: A buildpackage is a [distributable](https://github.com/buildpacks/spec/blob/main/distribution.md) artifact that contains a buildpack. +- Buildpackage: A buildpackage is a [distributable](https://github.com/buildpacks/spec/blob/main/distribution.md) artifact that contains a buildpack. # Motivation [motivation]: #motivation @@ -32,16 +34,15 @@ There is a limit in the number of layer an image can have, at least on Docker, w - What use cases does it support? -Buildpacks provider like Paketo have composite buildpacks with several layers, when they pull many of those together into a builder, hitting the layer limit for a container image happens very often. A feature for the Buildpack author to group the buildpacks by any attribute will allow them to squash those groups into one layer and reduce their total number of layers, avoiding the layer limit. +Buildpacks provider like Paketo have Composite Buildpacks with several layers, when they pull many of those together into a Builder, hitting the layer limit for a container image happens very often. A feature for the Builder author to group the Buildpacks by any attribute will allow them to squash those groups into one layer and reduce their total number of layers, avoiding the layer limit. - What is the expected outcome? -When Buildpacks Authors execute commands like: +When Builder Authors execute the command: -`pack builder create ... ` or -`pack buildpack package ... ` +`pack builder create ... ` -The final OCI image artifact (A) SHOULD contain layers blobs with more than *one* buildpack according to the configuration provided by the user. If we compare an artifact (B) created *without* `` then: +The final Builder (A) SHOULD contain layers blobs with more than *one* buildpack according to the configuration provided by the user. If we compare an artifact (B) created *without* `` then: $numberOfBuildpackLayers(A) \leq numberOfBuildpackLayers(B)$ @@ -51,26 +52,25 @@ A and B MUST be otherwise interchangeable, only differing by their number of lay # What it is [what-it-is]: #what-it-is -The proposal is to include new experimental flags to the following commands on pack: +The proposal is to include a new experimental flag to the following command on Pack: - `pack builder create` -- `pack buildpack package` -The new flags will move from experimental status to supported status when maintainers deem it appropriate. +The new flag will move from experimental status to supported status when maintainers deem it appropriate. -The new flags to be included are: +The new flag to be included is: - `--flatten=` will flatten the Buildpacks specified after the `flatten` flag into a single layer. Can be used more than once, with each use resulting in a single layer. -We also need to define how a platform implementor needs to consume a flatten buildpackpage or builder. +We also need to define how a Platform implementor needs to consume a flattened Builder. -- When a platform consumes an OCI image artifact, they will need to inspect each buildpack layer blob and determine if the blob contains more than one buildpack, in such as case, they will need to process those buildpacks correctly. +- When a Platform consumes a Builder, they will need to inspect each Buildpack layer blob and determine if the blob contains more than one Buildpack, in such as case, they will need to process those Buildpacks correctly. # How it Works [how-it-works]: #how-it-works -Let's say we have a composite buildpack (CB1) with the following dependency tree: +Let's say we have a Composite Buildpack (CB1) with the following dependency tree: ```mermaid flowchart TD A[CB1] @@ -84,7 +84,7 @@ flowchart TD C --> BPE[BP4] ``` -Until now, when a buildpack like this is being shipped into an OCI image every individual buildpack is being saved in one layer, as a result we will have: +Until now, when a Buildpack like this is being shipped into a Builder every individual Buildpack is being saved in one layer, as a result we will have: $$ layer_1 = [CB_1] \\ @@ -97,12 +97,12 @@ layer_7 = [BP_3] \\ total = \text{7 layers} $$ -Noticed that duplicated buildpacks are cleaned up. +Noticed that duplicated Buildpacks are cleaned up. -We can use the new `flatten` flag to reduce the number of OCI image layers used by the buildpacks in different ways. +We can use the new `flatten` flag to reduce the number of Builder layers used by the buildpacks in different ways. * `--flatten=` i.e. `--flatten= --flatten=`: - Will group the given buildpacks into one layer and keep the other ones as single layers buildpacks, the result will be: + Will group the given Buildpacks into one layer and keep the other ones as single layers Buildpacks, the result will be: ```mermaid classDiagram @@ -136,6 +136,7 @@ $$ # Migration [migration]: #migration + The current [distribution spec](https://github.com/buildpacks/spec/blob/main/distribution.md#buildpackage) defines: ``` @@ -144,7 +145,7 @@ Each buildpack layer blob MUST contain a single buildpack at the following file /cnb/buildpacks/// ``` -A Buildpackage and a Builder flattened with this new feature would not be consumable by older platform implementations because they are not expecting to find more than one buildpack on a blob layer. +A Builder flattened with this new feature would not be consumable by older platform implementations because they are not expecting to find more than one buildpack on a blob layer. +- Status: Approved - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) @@ -110,7 +110,7 @@ classDiagram CB1 } class Layer2 { - G1 + G1 } class Layer3 { BP1 @@ -182,7 +182,7 @@ Discuss prior art, both the good and bad. --- -% \ No newline at end of file +--->% diff --git a/text/0124-pack-manifest-list-commands.md b/text/0124-pack-manifest-list-commands.md index 65fc80059..0e2a7f5a0 100644 --- a/text/0124-pack-manifest-list-commands.md +++ b/text/0124-pack-manifest-list-commands.md @@ -3,7 +3,7 @@ - Name: Manifest List Commands for Pack - Start Date: 2023-04-19 - Author(s): [Juan Bustamante](https://github.com/jjbustamante) -- Status: Draft +- Status: Approved - RFC Pull Request: (leave blank) - CNB Pull Request: (leave blank) - CNB Issue: (leave blank) @@ -22,7 +22,7 @@ This RFC proposes to create a new set of CRUD commands in pack to handle manifes [definitions]: #definitions - Image Manifest: The image manifest provides a configuration and set of layers for a single container image for a specific architecture and operating system. See [spec](https://github.com/opencontainers/image-spec/blob/main/manifest.md) -- Image Index: The image index is a higher-level manifest which points to specific image manifests, ideal for one or more platforms. See [spec](https://github.com/opencontainers/image-spec/blob/main/image-index.md) +- Image Index: The image index is a higher-level manifest which points to specific image manifests, ideal for one or more platforms. See [spec](https://github.com/opencontainers/image-spec/blob/main/image-index.md) # Motivation [motivation]: #motivation @@ -38,10 +38,10 @@ Or the conversations around this topic in our [slack channel](https://cloud-nati - What use cases does it support? -Currently, buildpack authors can build and package their buildpacks for different OS and Architectures, but when they distribute them the URI for a buildpack can’t disambiguate, they need to use different tags to differentiate between them. This makes harder for users to consume those Buildpacks. +Currently, buildpack authors can build and package their buildpacks for different OS and Architectures, but when they distribute them the URI for a buildpack can’t disambiguate, they need to use different tags to differentiate between them. This makes harder for users to consume those Buildpacks. The solution is to share a single URI for all the different OS and Architectures, and the way to do that is using a manifest list. -Adding commands to support the operations to handle the manifest list will allow buildpack authors to migrate their existing buildpacks to support multi-arch, without afecting their current existing process. +Adding commands to support the operations to handle the manifest list will allow buildpack authors to migrate their existing buildpacks to support multi-arch, without afecting their current existing process. - What is the expected outcome? @@ -62,17 +62,17 @@ The proposal is to add a new _experimental_ command `pack manifest` and differen - `pack manifest push` will push a manifest list to a registry - `pack manifest inspect` will show the manifest information stored in local storage -Our target user affected by the feature is: **Buildpack Authors**. Let's see some examples of how this feature will work. +Our target user affected by the feature is: **Buildpack Authors**. Let's see some examples of how this feature will work. -Currently, if we check [sample-packages](https://hub.docker.com/r/cnbs/sample-package/tags) at dockerhub we will notice that we have a composed buildpack called `hello-universe` and we offer two tags to support different architectures: -- `cnbs/sample-package:hello-universe` for linux and -- `cnbs/sample-package:hello-universe-windows`. +Currently, if we check [sample-packages](https://hub.docker.com/r/cnbs/sample-package/tags) at dockerhub we will notice that we have a composed buildpack called `hello-universe` and we offer two tags to support different architectures: +- `cnbs/sample-package:hello-universe` for linux and +- `cnbs/sample-package:hello-universe-windows`. Let's suppose our linux version is called `cnbs/sample-package:hello-universe-linux` to keep the same naming convention, but we will keep it as it is for simplicity. If we want to distribute the `hello-universe` buildpack for any **architecture/os/variant** combination we need to use a tool outside the CNB ecosystem to create a manifest list. With the proposed experimental commands on pack we can do: ```bash -$ pack manifest create cnbs/sample-package:hello-multiarch-universe \ - cnbs/sample-package:hello-universe \ +$ pack manifest create cnbs/sample-package:hello-multiarch-universe \ + cnbs/sample-package:hello-universe \ cnbs/sample-package:hello-universe-windows ``` @@ -100,11 +100,11 @@ By default, the command will create a manifest list in the local storage using t "os": "windows", "architecture": "" } - } + } ] } ``` -The idea to save the manifest list locally is to allow the user to update the manifest before pushing it to a registry, +The idea to save the manifest list locally is to allow the user to update the manifest before pushing it to a registry, in this case, we need to define the **architecture** field because it is empty in our examples. We can use the `pack manifest annotate` command to add the architecture information: @@ -167,7 +167,7 @@ The proposal is to implement an abstraction of an OCI *Image Index* and expose i ## Image Index Abstraction -A new high level abstraction to represent an OCI Image Index is proposed, similar to the [Image](https://github.com/buildpacks/imgutil/blob/main/image.go) interface exposed in *imgutil* repository, +A new high level abstraction to represent an OCI Image Index is proposed, similar to the [Image](https://github.com/buildpacks/imgutil/blob/main/image.go) interface exposed in *imgutil* repository, we proposed a new *ManifestList* interface to expose the behavior of an OCI Image Index. ```mermaid @@ -182,21 +182,21 @@ classDiagram } class remote_ManifestList { - +NewManifestList(repoName string, keychain authn.Keychain) ManifestList + +NewManifestList(repoName string, keychain authn.Keychain) ManifestList } class local_ManifestList { +NewManifestList(repoName string, path string) ManifestList } - + class AnnotateFields { +String Architecture +String OS +String Variant } - + ManifestList <|-- remote_ManifestList - ManifestList <|-- local_ManifestList + ManifestList <|-- local_ManifestList ``` Two implementations: *remote* and *local* are proposed, *remote* will take care of implementing the interaction with an OCI registry and *local* will deal with the local storage operations. @@ -213,7 +213,7 @@ Using a [C4 component diagram](https://c4model.com/#ComponentDiagram), we can de ### Considerations -#### When a user wants to create a manifest list using a manifest outside the user's repo. +#### When a user wants to create a manifest list using a manifest outside the user's repo. As a pack user I want to create a manifest list `foo/my-manifest:my-tag` using a manifest outside my repository `foo` for example `other/external-manifest:latest`. @@ -258,7 +258,7 @@ Flags: -r, --registry string Registry URL to publish the image index ``` -#### Annotate (os/arch) a Manifest List +#### Annotate (os/arch) a Manifest List Sometimes a manifest list could reference an image that doesn't specify the *architecture*, for example, [check](https://hub.docker.com/r/cnbs/sample-package/tags) our sample buildpack packages. The `annotate` command allows users to update those values before pushing the manifest list a registry @@ -377,7 +377,7 @@ One important concern for users is to inspect the content of a multi-arch builde - `pack builder inspect` - `pack buildpack inspect` -The `--platform` flag specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). By default it will reference the host values. +The `--platform` flag specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). By default it will reference the host values. The output of the commands should remain the same. @@ -412,7 +412,7 @@ The impact of not doing this is that users will need to use other tools to creat [prior-art]: #prior-art These features are inspired in similar commands in other tools like: -- [docker](https://docs.docker.com/engine/reference/commandline/manifest/) +- [docker](https://docs.docker.com/engine/reference/commandline/manifest/) - [podman](https://docs.podman.io/en/v3.2.0/markdown/podman-manifest-create.1.html) # Unresolved Questions From a68ed1dae7b127aa59458c61d050423588be5d10 Mon Sep 17 00:00:00 2001 From: Kritka Sahni <122665407+kritkasahni-google@users.noreply.github.com> Date: Tue, 17 Oct 2023 16:03:56 -0700 Subject: [PATCH 363/372] Create 0000-creator-skip-sbom.md Signed-off-by: Kritka Sahni <122665407+kritkasahni-google@users.noreply.github.com> --- text/0000-creator-skip-sbom.md | 131 +++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 text/0000-creator-skip-sbom.md diff --git a/text/0000-creator-skip-sbom.md b/text/0000-creator-skip-sbom.md new file mode 100644 index 000000000..6e01fcf56 --- /dev/null +++ b/text/0000-creator-skip-sbom.md @@ -0,0 +1,131 @@ +# Meta +[meta]: #meta +- Name: Enable CNB_SKIP_SBOM IN /cnb/lifecycle/creator +- Start Date: (fill in today's date: 2023-10-17) +- Author(s): kritkasahni-google +- Status: Draft +- RFC Pull Request: +- CNB Pull Request: +- CNB Issue: +- Supersedes: N/A + +# Summary +[summary]: #summary + +Enable CNB_SKIP_SBOM IN /cnb/lifecycle/creator to skip restoring SBOM layer from previous app image. We support CNB_SKIP_LAYERS in analyzer which does the same thing and we should support the same in creator also. + +# Definitions +[definitions]: #definitions +* lifecycle: software that orchestrates a CNB build. +* creator: executes all the lifecycle phases one by one in order. +* analyzer: lifecycle phase that restores SBOM layer from previous app image. +* restorer: lifecycle phase that restores layers from cache. +* SBOM: a software bill of materials (SBOM) is a list of all the components that make up the app image. + +# Motivation +[motivation]: #motivation + +To skip restoring SBOM layer from previous image when platform executes lifecycle by calling /cnb/lifecycle/creator. Restoring SBOM layer from previous app image can cause degraded build latency but if buildpack logic does not rely on SBOM from previous app image then should be able to skip restoring it. + +# What it is +[what-it-is]: #what-it-is + +CNB_SKIP_LAYERS is used by /cnb/lifecycle/analyzer to skip restoring SBOM layer from previous app image. +Need a similar mechanism for /cnb/lifecyle/creator specifically to skip restoring only the SBOM layer. + +The target personas affected by this change are: + + - **buildpack user**: if buildpacks don't rely on reusing SBOM layer then buildpack user should ideally see improved build latency by skipping SBOM restoration but reusing other layers from previous app image. + - **Platform implementors**: they will choose to skip restoring SBOM by providing CNB_SKIP_SBOM to trigger /cnb/lifecycle/creator. + + +# How it Works +[how-it-works]: #how-it-works + +Similar to how CNB_SKIP_LAYERS is handled in analyzer whether SBOM needs to be [restored](https://github.com/buildpacks/lifecycle/blob/292aa492a72f4e180bb92d109a73ebf7c8a0451d/phase/analyzer.go#L38) or [not](https://github.com/buildpacks/lifecycle/blob/292aa492a72f4e180bb92d109a73ebf7c8a0451d/phase/analyzer.go#L30) today, CNB_SKIP_SBOM will be be handled in same way in analyzer. +At the platform level, it would be input same way as CNB_SKIP_LAYERS [here](https://github.com/buildpacks/lifecycle/blob/292aa492a72f4e180bb92d109a73ebf7c8a0451d/platform/defaults.go#L184) and [handled](https://github.com/buildpacks/lifecycle/blob/main/platform/lifecycle_inputs.go#L82) like:- + + +``` + var skipSBOM bool + if boolEnv(EnvSkipSBOM){ + skipSBOM = true + } +``` + +In the analyzer, + +``` +analyzer := &Analyzer{ + Logger: logger, + SBOMRestorer: &layer.NopSBOMRestorer{}, + PlatformAPI: f.platformAPI, + } + + ... + if f.platformAPI.AtLeast("0.8") && !inputs.SkipLayers && !inputs.SkipSBOM { + analyzer.SBOMRestorer = &layer.DefaultSBOMRestorer{ + LayersDir: inputs.LayersDir, + Logger: logger, + } + } +``` + +# Migration +[migration]: #migration + +CNB_SKIP_SBOM/ will be an optional input to /cnb/lifecycle/creator, and will be false by default. We maybe need to add a new API option for buildpack users, to choose whether this should be enabled. + +# Drawbacks +[drawbacks]: #drawbacks + +N/A + +# Alternatives +[alternatives]: #alternatives + +Platforms that execute lifecycle today via /cnb/lifecycle/creator are unable to skip restoring SBOM layer from previous app image unless they skip reusing previous app image entirely. + +# Prior Art +[prior-art]: #prior-art + +We already support enabling CNB_SKIP_LAYERS in /cnb/lifecycle/analyzer and /cnb/lifecycle/restorer, and CNB_SKIP_RESTORE in /cnb/lifecycle/creator. +* CNB_SKIP_LAYERS in /cnb/lifecycle/analyzer to skip restoring SBOM from previous app image. +* CNB_SKIP_LAYERS in /cnb/lifecycle/restorer to skip reusing previous app image layers entirely. +* CNB_SKIP_RESTORE in /cnb/lifecycle/creator to skips restoring SBOM plus all other layers entirely from previous app image. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +N/A + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes +This new feature will affect the API of [Create](https://buildpacks.io/docs/concepts/components/lifecycle/create/) phase by adding the following fields. + +Back to API changes, we will add a new flag to control this. + +| Input | Environment Variable | DefaultValue | Description | +|----------------|-----------------------|--------------|----------------------------------------------| +| `` | `CNB_SKIP_SBOM` | `false` | Skip SBOM restoration | + +# History +[history]: #history + + From 9752e5939fd1a9111cba903cdd8e8cf2a0d2c8a9 Mon Sep 17 00:00:00 2001 From: Natalie Arellano Date: Mon, 23 Oct 2023 10:15:57 -0400 Subject: [PATCH 364/372] RFC 0125 [#291] Signed-off-by: Natalie Arellano --- ...parallel-export.md => 0125-lifecycle-parallel-export.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename text/{0000-lifecycle-parallel-export.md => 0125-lifecycle-parallel-export.md} (98%) diff --git a/text/0000-lifecycle-parallel-export.md b/text/0125-lifecycle-parallel-export.md similarity index 98% rename from text/0000-lifecycle-parallel-export.md rename to text/0125-lifecycle-parallel-export.md index bc5cbca3a..0d193a8ab 100644 --- a/text/0000-lifecycle-parallel-export.md +++ b/text/0125-lifecycle-parallel-export.md @@ -3,10 +3,10 @@ - Name: Export App Image and Cache Image in Parallel - Start Date: 2023-08-26 - Author(s): ESWZY -- Status: Draft -- RFC Pull Request: +- Status: Approved +- RFC Pull Request: [rfcs#291](https://github.com/buildpacks/rfcs/pull/291) - CNB Pull Request: [lifecycle#1167](https://github.com/buildpacks/lifecycle/pull/1167) -- CNB Issue: +- CNB Issue: N/A - Supersedes: N/A # Summary From 91a8eaac4204a89c9301fe836785a17f80ba3336 Mon Sep 17 00:00:00 2001 From: Pavel Busko Date: Mon, 9 Oct 2023 14:41:32 +0200 Subject: [PATCH 365/372] Add RFC for extension layers Co-authored-by: Ralf Pannemans Signed-off-by: Ralf Pannemans Co-authored-by: Pavel Busko Signed-off-by: Pavel Busko --- text/0000-extension-layer.md | 90 ++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 text/0000-extension-layer.md diff --git a/text/0000-extension-layer.md b/text/0000-extension-layer.md new file mode 100644 index 000000000..e337fb0b0 --- /dev/null +++ b/text/0000-extension-layer.md @@ -0,0 +1,90 @@ +# Meta +[meta]: #meta +- Name: Add extension layer to exchange data +- Start Date: 2023-10-09 +- Author(s): [c0d1ngm0nk3y](https://github.com/c0d1ngm0nk3y), [pbusko](https://github.com/pbusko) +- Status: Draft +- RFC Pull Request: +- CNB Pull Request: +- CNB Issue: +- Related: [RFC#0105 Support Dockerfiles](https://github.com/buildpacks/rfcs/blob/main/text/0105-dockerfiles.md) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This RFC introduces support for Extension Layers to allow data transfer between the build environment and the Kaniko execution. + +# Motivation +[motivation]: #motivation + +This change allows extensions to possess their own layers to utilize during the generation/extend process. Additionally, it ensures that extension output does not inadvertently interfere with other extension or buildpack layers during the build, and it does not unintentionally become part of the final application image. + +This would allow distroless run images to be extended. + +# What it is +[what-it-is]: #what-it-is + +This follows up on RFC-0105 and proposes that during the execution of the extension's `bin/generate`, an extension is allowed to write arbitrary data to its exclusive layer. This data then becomes accessible during the execution of the `extend` phase via Kaniko context. The content of these extension-specific layers is ignored at build and launch time, it serves only the extension phase. + +# How it Works +[how-it-works]: #how-it-works + +Before execution of the `bin/generate`, the lifecycle will create a distinct writable layer for each extension which passed detection. The extensions can then write to these layers, and the Kaniko context is set to the corresponding layer during the `extend` phase instead of the `` directory. The Extension Layers will not be included in the final image by the lifecycle. + +The location of the Extension Layer will be provided to the `bin/generate` via additional environment variable `CNB_EXT_LAYER_DIR`. + +# Migration +[migration]: #migration + +Since we would change the Kaniko context, this would be breaking for existing extensions. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +The workspace would no longer be the Kaniko context. But the benefit of having it in the first place seems quite limited anyway. + +# Alternatives +[alternatives]: #alternatives + +- Allow multi-stage Dockerfiles + +# Prior Art +[prior-art]: #prior-art + +Discuss prior art, both the good and bad. + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- Should the `bin/generate` be executed during the `extend` phase instead of the `detect` phase? +- The Kaniko context might consist of two folders: `` and the Extension Layer for better compatibility. + +# Spec. Changes (OPTIONAL) +[spec-changes]: #spec-changes + +This RFC requires changes to the layers metadata and the `extend` phase: + +- layer metadata needs ti ubducate if a layer is a "extension layer" +- env variable with the layer location for the extension to write to +- kaniko context should be the extension layer and not the workspace + + \ No newline at end of file From 54fd7b04d74ab771cc5f158d4532beeb1add3063 Mon Sep 17 00:00:00 2001 From: Pavel Busko Date: Wed, 25 Oct 2023 14:24:19 +0200 Subject: [PATCH 366/372] add examples Signed-off-by: Pavel Busko --- text/0000-extension-layer.md | 48 ++++++++++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 10 deletions(-) diff --git a/text/0000-extension-layer.md b/text/0000-extension-layer.md index e337fb0b0..6fc860499 100644 --- a/text/0000-extension-layer.md +++ b/text/0000-extension-layer.md @@ -25,26 +25,56 @@ This would allow distroless run images to be extended. # What it is [what-it-is]: #what-it-is -This follows up on RFC-0105 and proposes that during the execution of the extension's `bin/generate`, an extension is allowed to write arbitrary data to its exclusive layer. This data then becomes accessible during the execution of the `extend` phase via Kaniko context. The content of these extension-specific layers is ignored at build and launch time, it serves only the extension phase. +This follows up on RFC-0105 and proposes that during the execution of the extension's `./bin/generate`, an extension is allowed to write arbitrary data to the `context` folder within its exclusive layer. This data then becomes accessible during the execution of the `extend` phase via Kaniko build context. The content of these extension-specific layers is ignored at build and launch time, it serves only the extension phase. # How it Works [how-it-works]: #how-it-works -Before execution of the `bin/generate`, the lifecycle will create a distinct writable layer for each extension which passed detection. The extensions can then write to these layers, and the Kaniko context is set to the corresponding layer during the `extend` phase instead of the `` directory. The Extension Layers will not be included in the final image by the lifecycle. +- New root directory `/layers-ext` is introduced which contains extension layers. +- Before execution of the `./bin/generate`, the lifecycle will create a distinct writable layer `/layers-ext/` for each extension which passed detection. +- The `/layers-ext/` is provided to the `./bin/generate` as `` directory. +- In addition to the files specified in [RFC#0105](https://github.com/buildpacks/rfcs/blob/main/text/0105-dockerfiles.md), the extension may create the `/context` folder with an arbitrary content. +- If the folder `/context` is present it will be set as Kaniko build context during the `extend` phase instead of the `` directory. +- If the folder `/context` is not present, Kaniko build context defaults to the `` folder. + +The `/layers-ext` will not be included in the final image by the lifecycle. -The location of the Extension Layer will be provided to the `bin/generate` via additional environment variable `CNB_EXT_LAYER_DIR`. +### Example: Extend distroless run image with Debian packages. + +This example extension would allow to install `tar` package on the run image without package manager (distroless image). The extension contains `./bin/generate` and `./bin/custom-installer` file, which installs `.deb` files. + +##### `./bin/generate` + +```bash +#!/bin/sh + +mkdir -p ${CNB_OUTPUT_DIR}/context + +cp ${CNB_EXTENSION_DIR}/bin/custom-installer ${CNB_OUTPUT_DIR}/context/ +curl -o ${CNB_OUTPUT_DIR}/context/tar.deb http://security.ubuntu.com/ubuntu/pool/main/t/tar/tar_1.34+dfsg-1ubuntu0.1.22.04.1_amd64.deb + +cat >> "${CNB_OUTPUT_DIR}/run.Dockerfile" <` and the Extension Layer for better compatibility. +- Should the `./bin/generate` be executed during the `extend` phase instead of the `detect` phase? # Spec. Changes (OPTIONAL) [spec-changes]: #spec-changes This RFC requires changes to the layers metadata and the `extend` phase: -- layer metadata needs ti ubducate if a layer is a "extension layer" -- env variable with the layer location for the extension to write to -- kaniko context should be the extension layer and not the workspace +- allow optional folder `/context` with an arbitrary content to be provided by extension. +- if the `/context` is present, kaniko context should be set to this folder instead of the ``. +- Status: Approved - RFC Pull Request: - CNB Pull Request: - CNB Issue: