diff --git a/README.md b/README.md index b04d99d7..f30247a5 100644 --- a/README.md +++ b/README.md @@ -30,4 +30,4 @@ An implementation is compliant if it satisfies all the MUST, MUST NOT, REQUIRED, These documents currently specify: #### Buildpack API: 0.2 -#### Platform API: 0.2 +#### Platform API: 0.3 diff --git a/platform.md b/platform.md index 46e363c9..a8940d67 100644 --- a/platform.md +++ b/platform.md @@ -7,60 +7,116 @@ A platform orchestrates a lifecycle to make buildpack functionality available to Examples of a platform might include: 1. A local CLI tool that uses buildpacks to create OCI images -2. A plugin for a continuous integration service that uses buildpacks to create OCI images -3. A cloud application platform that uses buildpacks to build source code before deployment +1. A plugin for a continuous integration service that uses buildpacks to create OCI images +1. A cloud application platform that uses buildpacks to build source code before deployment ## Table of Contents - + - [Platform Interface Specification](#platform-interface-specification) - [Table of Contents](#table-of-contents) - [Platform API Version](#platform-api-version) - - [Compatibility Verification](#compatibility-verification) + - [Terminology](#terminology) + - [CNB Terminology](#cnb-terminology) + - [Additional Terminology](#additional-terminology) - [Stacks](#stacks) - [Compatibility Guarantees](#compatibility-guarantees) - [Build Image](#build-image) - [Run Image](#run-image) - [Mixins](#mixins) + - [Lifecycle Interface](#lifecycle-interface) + - [Platform API Compatibility](#platform-api-compatibility) + - [Operations](#operations) + - [Build](#build) + - [Rebase](#rebase) + - [Launch](#launch) + - [Usage](#usage) + - [`detector`](#detector) + - [`analyzer`](#analyzer) + - [Layer analysis](#layer-analysis) + - [`restorer`](#restorer) + - [Layer restoration](#layer-restoration) + - [`builder`](#builder) + - [`exporter`](#exporter) + - [`creator`](#creator) + - [`rebaser`](#rebaser) + - [`launcher`](#launcher) + - [Process Selection](#process-selection) + - [Run Image Resolution](#run-image-resolution) + - [Registry Authentication](#registry-authentication) - [Buildpacks](#buildpacks) - [Buildpacks Directory Layout](#buildpacks-directory-layout) - [Security Considerations](#security-considerations) - [Additional Guidance](#additional-guidance) - [Environment](#environment) - - [Run Image Rebasing](#run-image-rebasing) + - [Buildpack Environment](#buildpack-environment) + - [Stack-Provided Variables](#stack-provided-variables) + - [User-Provided Variables](#user-provided-variables) + - [Launch Environment](#launch-environment) - [Caching](#caching) + - [Build Reproducibility](#build-reproducibility) - [Data Format](#data-format) - - [order.toml (TOML)](#ordertoml-toml) - - [group.toml (TOML)](#grouptoml-toml) + - [Files](#files) + - [`analyzed.toml` (TOML)](#analyzedtoml-toml) + - [`group.toml` (TOML)](#grouptoml-toml) + - [`metadata.toml` (TOML)](#metadatatoml-toml) + - [`order.toml` (TOML)](#ordertoml-toml) + - [`plan.toml` (TOML)](#plantoml-toml) + - [`project-metadata.toml` (TOML)](#project-metadatatoml-toml) + - [`stack.toml` (TOML)](#stacktoml-toml) + - [Labels](#labels) + - [`io.buildpacks.build.metadata` (JSON)](#iobuildpacksbuildmetadata-json) + - [`io.buildpacks.lifecycle.metadata` (JSON)](#iobuildpackslifecyclemetadata-json) + - [`io.buildpacks.project.metadata` (JSON)](#iobuildpacksprojectmetadata-json) ## Platform API Version -The Platform API version: +This document specifies Platform API version `0.3`. + +Platform API versions: - MUST be in form `.` or ``, where `` is equivalent to `.0` - - MUST describe the implemented Platform API. - - SHALL indicate compatibility with a given lifecycle according to the following rules: - - When `` is `0`, the platform is only compatible with lifecycles implementing that exact Platform API. - - When `` is greater than `0`, the platforms is only compatible with lifecycles implementing platform API - `.`, where `` of the lifecycle equals `` of the platform and `` of the lifecycle - is greater than or equal to `` of the platform. + - When `` is greater than `0` increments to `` SHALL exclusively indicate additive changes -### Compatibility Verification +## Terminology -The lifecycle SHALL verify compatibility if the environment variable `CNB_PLATFORM_API` is set. The value of this -environment variable MUST be the version of the Platform API the platform implements. Compatibility verification SHALL -NOT occur if this environment variable is not set. Compatibility verification SHALL occur before any other validation. +#### CNB Terminology +A **buildpack** refers to software compliant with the [Buildpack Interface Specification](https://github.com/buildpacks/spec/blob/main/buildpack.md). -## Stacks +A **stack** is a contract defined by a **build image** and a **run image** that are only updated with security patches. + +A **build image** is an OCI image that provides the base of the **build environment**. + +A **run image** is an OCI image that provides the base from which **app images** are built. + +The **build environment** refers to the containerized environment in which the lifecycle executes buildpacks. + +A **mixin** is a named set of additions to a stack that can be used to make additive changes to the contract. + +An **app image** refers to an OCI image generated by the lifecycle by extending the **run image** with any or all of the following: **app layers**, **launch layers**, **launcher layers**, image configuration. + +A **launch layer** refers to a layer in the app image created from a `/` directory as specified in the [Buildpack Interface Specification](buildpack.md). -A **stack** is a contract defined by a base run OCI image and a base build OCI image that are only updated with security patches. -Stack images can be modified with mixins in order to make additive changes to the contract. +An **app layer** refers to a layer in the app image created from the `` directory as specified in the [Buildpack Interface Specification](buildpack.md). -A **mixin** is a named set of additions to a stack that avoid changing the behavior of buildpacks or apps that do not depend on the mixin. +A **run image layer** refers to a layer in the **app image** originating from the **run image**. -A **launch layer** refers to a layer in the app OCI image created from a `/` directory as specified in the [Buildpack Interface Specification](buildpack.md). +A **launcher layer** refers to a layer in the app OCI image containing the **launcher** itself and/or launcher configuration. -An **app layer** refers to a layer created from the `` directory as specified in the [Buildpack Interface Specification](buildpack.md). +The **launcher** refers to a lifecycle executable packaged in the **app image** for the purpose of executing processes at runtime. +#### Additional Terminology +An **image reference** refers to either a **tag reference** or **digest reference**. + +A **tag reference** refers to an identifier of form `/:` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. + +A **digest reference** refers to a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) identifier of form `/@` which locates an image manifest in an [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec/blob/master/spec.md) compliant registry. + +The following is a non-exhaustive list of terms defined in the [OCI Image Format Specification](https://github.com/opencontainers/image-spec) used throughout this document: +* **image config** https://github.com/opencontainers/image-spec/blob/master/config.md#oci-image-configuration +* **imageID** - https://github.com/opencontainers/image-spec/blob/master/config.md#imageid +* **diffID** - https://github.com/opencontainers/image-spec/blob/master/config.md#layer-diffid + +## Stacks ### Compatibility Guarantees Stack image authors SHOULD ensure that build image versions maintain [ABI-compatibility](https://en.wikipedia.org/wiki/Application_binary_interface) with previous versions, although violating this requirement will not change the behavior of previously built images containing app and launch layers. @@ -74,8 +130,6 @@ During build, platforms MUST use the same set of mixins for the run image as wer ### Build Image -The platform MUST execute the detection and build phases of the lifecycle on the build image. - The platform MUST ensure that: - The image config's `User` field is set to a non-root user with a writable home directory. @@ -85,37 +139,8 @@ The platform MUST ensure that: - The image config's `Label` field has the label `io.buildpacks.stack.id` set to the stack ID. - The image config's `Label` field has the label `io.buildpacks.stack.mixins` set to a JSON array containing mixin names for each mixin applied to the image. -To initiate the detection phase, the platform MUST invoke the `/cnb/lifecycle/detector` executable with the user and environment defined in the build image config. -Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. - -```bash -/cnb/lifecycle/detector -buildpacks /cnb/by-id -order /cnb/order.toml -group ./group.toml -plan ./plan.toml -``` - -Where: - -- `-buildpacks` MUST specify input from a buildpacks directory as defined in the [Buildpacks Directory Layout](#buildpacks-directory-layout) section. -- `-order` MUST specify input from an overriding `order.toml` file path as defined in the [Data Format](#data-format) section. -- `-group` MUST specify output to a `group.toml` file path as defined in the [Data Format](#data-format) section. -- `-plan` MUST specify output to a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). - -To initiate the build phase, the platform MUST invoke the `/cnb/lifecycle/builder` executable with the user and environment defined in the build image config. -Invoking this executable with no flags is equivalent to the following invocation including all accepted flags and their default values. - -```bash -/cnb/lifecycle/builder -buildpacks /cnb/by-id -group ./group.toml -plan ./plan.toml -``` - -Where: - -- `-buildpacks` MUST specify input from a buildpacks directory as defined in the [Buildpacks Directory Layout](#buildpacks-directory-layout) section. -- `-group` MUST specify input from a `group.toml` file path as defined in the [Data Format](#data-format) section. -- `-plan` MUST specify input from a Build Plan as defined in the [Buildpack Interface Specification](buildpack.md). - ### Run Image -The platform MUST provide the lifecycle with a reference to the run image during the export phase. - The platform MUST ensure that: - The image config's `User` field is set to a user with the same UID and primary GID as in the build image. @@ -136,6 +161,453 @@ A platform MAY support any number of mixins for a given stack in order to suppor Changes introduced by mixins SHOULD be restricted to the addition of operating system software packages that are regularly patched with strictly backwards-compatible security fixes. However, mixins MAY consist of any changes that follow the [Compatibility Guarantees](#compatibility-guarantees). +## Lifecycle Interface +### Platform API Compatibility + +The platform SHOULD set `CNB_PLATFORM_API=` in the lifecycle's execution environment + +If `CNB_PLATFORM_API` is set in the lifecycle's execution environment, the lifecycle: + - MUST either conform to the matching version of this specification or + - MUST fail if it does not support `` + +### Operations + +#### Build +A single app image build* consists of the following phases: + +1. Detection +1. Analysis +1. Cache Restoration +1. Build* +1. Export + +A platform MUST execute these phases either by invoking the following phase-specific lifecycle binaries in order: +1. `/cnb/lifecycle/detector` +1. `/cnb/lifecycle/analyzer` +1. `/cnb/lifecycle/restorer` +1. `/cnb/lifecycle/builder` +1. `/cnb/lifecycle/exporter` + +or by executing `/cnb/lifecycle/creator`. + +> \* **build** is an overloaded term that refers to both a single phase and the operation comprised of the above phases. +> The meaning of any particular instance of the word **build** must be assessed in context + +#### Rebase +When an updated run image with the same stack ID is available, an updated app image SHOULD be generated from the existing app image config by replacing the run image layers in the existing app image with the layers from the new run image. +This is referred to as rebasing the app, launch, and launcher layers onto the new run image layers. +When layers are rebased, any app image metadata referenceing to the original run image MUST be updated to reference to the new run image. +This entire operation is referred to as rebasing the app image. + +Rebasing allows for fast runtime OS-level dependency updates for app images without requiring a rebuild. A rebase requires minimal data transfer when the app and run images are colocated on a Docker registry that supports [Cross Repository Blob Mounts](https://docs.docker.com/registry/spec/api/#cross-repository-blob-mount). + +To rebase an app image a platform MUST execute the `/cnb/lifecycle/rebaser` or perform an equivalent operation. + +#### Launch +`/cnb/lifecycle/launcher` is responsible for launching user and buildpack provided processes in the correct execution environment. +`/cnb/lifecycle/launcher` SHALL be the `ENTRYPOINT` for all app images. + +### Usage + +All lifecycle phases: + +- MUST read `CNB_PLATFORM_API` from the execution environment and evaluate compatibility before attempting to parse other inputs (see [Platform API Compatibility](#platform-api-compatibility)) +- MUST give command line inputs precedence over other inputs + +#### `detector` +The platform MUST execute `detector` in the **build environment** + +Usage: +``` +/cnb/lifecycle/detector \ + [-app ] \ + [-buildpacks ] \ + [-group ] \ + [-log-level ] \ + [-order ] \ + [-plan ] \ + [-platform ] +``` + +| Input | Environment Variable| Default Value | Description +|---------------|---------------------|-----------------|---------------------- +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to output group definition +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| `` | `CNB_ORDER_PATH` | `./order.toml` | Path to order definition (see [`order.toml` (TOML)](#ordertoml-(toml))) +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to output resolved build plan +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Success (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Detected buildpack group (see [`group.toml`](#grouptoml-toml)) +| `` | Resolved Build Plan (see [`plan.toml`](#plantoml-toml)) + +The lifecycle: +- SHALL detect a single group from `` and write it to `` using the [detection process](buildpack.md#phase-1-detection) outlined in the Buildpack Interface Specification +- SHALL write the resolved build plan from the detected group to `` + +#### `analyzer` +Usage: +``` +/cnb/lifecycle/analyzer \ + [-analyzed ] \ + [-cache-dir ] \ + [-cache-image ] \ + [-daemon] \ # sets + [-gid ] \ + [-group ] \ + [-layers ] \ + [-log-level ] \ + [-skip-layers ] \ + [-uid ] \ + +``` + +| Input | Environment Variable | Default Value | Description +|----------------|-----------------------|-------------------|---------------------- +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to output analysis metadata (see [`analyzed.toml` (TOML)](#analyzedtoml-toml) +| `` | `CNB_CACHE_DIR` | | Location of cache, provided as a directory +| ``| `CNB_CACHE_IMAGE` | | Location of cache, provided as an image +| `` | `CNB_USE_DAEMON` | `false` | Analyze image from docker daemon +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml` (TOML)](#group.toml-(toml))) +| `` | | | Image reference to be analyzed (usually the result of the previous build) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| ``| `CNB_SKIP_LAYERS` | `false` | Do not perform layer analysis +| `` | `CNB_USER_ID` | | UID of the stack `User` + +- **If** `` is `false`, `` MUST be a valid image reference +- **If** `` is `true`, `` MUST be either a valid image reference or an imageID +- The lifecycle MUST accept valid references to non-existent images without error. + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Success (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Analysis metadata (see [`analyzed.toml`](#analyzedtoml-toml) +| `//.sha` | Files containing the diffID of each analyzed layer +| `//.toml` | Files containing the layer content metadata of each analyzed layer (see data format in [Buildpack Interface Specification]((buildpack.md))) + +- The lifecycle MUST write [analysis metadata](#analyzedtoml-toml) to `` if `` is accessible. +- **If** `` is `false` the lifecycle MUST analyze any app image layers or cached layers created by any buildpack present in the provided ``. +- **If** `` is `true` the lifecycle MUST NOT perform layer analysis. + +##### Layer analysis +When analyzing a given layer the lifecycle SHALL: +- **If** `build=true`, `cache=false`: + - Do nothing +- **Else if** `launch=true`: + - Write layer metadata read from the analyzed image to `//.toml` + - Write the layer diffID from the analyzed image to `//.sha` +- **Else if** `cache=true`: + - Write layer metadata read from the cache to `//.toml` + - Write the layer diffID from the cache to `//.sha` + +#### `restorer` +Usage: +``` +/cnb/lifecycle/restorer \ + [-cache-dir ] + [-cache-image ] + [-gid ] \ + [-group ] \ + [-layers ] \ + [-log-level ] \ + [-uid ] +``` + +| Input | Environment Variable | Default Value | Description +|----------------|-----------------------|-----------------|---------------------- +| `` | `CNB_CACHE_DIR` | | Path to a cache directory +| ``| `CNB_CACHE_IMAGE` | | Reference to a cache image in an OCI image registry +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| `` | `CNB_USER_ID` | | UID of the stack `User` +| `//.sha` || | Files containing the diffID of each analyzed layer +| `//.toml` || | Files containing the layer content metadata of each analyzed layer (see data format in [Buildpack Interface Specification]((buildpack.md))) + +| Output | Description +|------------------------------------|---------------------------------------------- +| [exit status] | Success (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `///*` | Restored layer contents + +##### Layer restoration +For each layer metadata file found in the `` directory, the lifecycle: +- MUST restore cached layer contents if the cache contains a layer with matching diffID +- MUST remove layer metadata if `cache=true` AND the cache DOES NOT contain a layer with matching diffID + +#### `builder` +The platform MUST execute `builder` in the **build environment** + +Usage: +``` +/cnb/lifecycle/builder \ + [-app ] \ + [-buildpacks ] \ + [-group ] \ + [-layers ] \ + [-log-level ] \ + [-plan ] \ + [-platform ] +``` + +| Input | Env | Default Value | Description +|----------------|-----------------------|-------------------|---------------------- +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_BUILDPACKS_DIR` | `/cnb/buildpacks` | Path to buildpacks directory (see [Buildpacks Directory Layout](#buildpacks-directory-layout)) +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group definition (see [`group.toml`](#grouptoml-toml)) +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layers directory +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| `` | `CNB_PLAN_PATH` | `./plan.toml` | Path to resolved build plan (see [`plan.toml`](#plantoml-toml)) +| `` | `CNB_PLATFORM_DIR` | `/platform` | Path to platform directory + +| Output | Description +|--------------------------------------------|---------------------------------------------- +| [exit status] | Success (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `//` | Layer contents (see [Buildpack Interface Specfication](buildpack.md) +| `//.toml` | Layer metadata (see [Buildpack Interface Specfication](buildpack.md) +| `/config/metadata.toml` | Build metadata (see [`metadata.toml`](#metadatatoml-toml)) + +- The lifecycle SHALL execute all buildpacks in the order defined in `` according process outlined in the [Buildpack Interface Specification](buildpack.md). +- The lifecycle SHALL add all invoked buildpacks to`/config/metadata.toml`. +- The lifecycle SHALL aggregate all `processes`, `slices` and BOM entries returned by buildpacks in `/config/metadata.toml`. + +#### `exporter` +Usage: +``` +/cnb/lifecycle/exporter \ + [-analyzed ] \ + [-app ] \ + [-cache-dir ] \ + [-cache-image ] \ + [-daemon] \ # sets + [-gid ] \ + [-group ] \ + [-launch-cache ] \ + [-launcher ] \ + [-layers ] \ + [-log-level ] \ + [-process-type ] \ + [-project-metadata ] \ + [-run-image | -image ] \ # -image is Deprecated + [-stack ] \ + [-uid ] \ + [...] +``` + +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|---------------------|--------------------------------------- +| `` | `CNB_ANALYZED_PATH` | `./analyzed.toml` | Path to analysis metadata (see [`analyzed.toml` (TOML)](#analyzed.toml-(toml)) +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_CACHE_DIR` | | Path to a cache directory +| `` | `CNB_CACHE_IMAGE` | | Reference to a cache image in an OCI image registry +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` +| `` | `CNB_GROUP_PATH` | `./group.toml` | Path to group file (see [`group.toml`](#group.toml-(toml))) +| `` | | | Tag reference to which the app image will be written +| `` | `CNB_LAUNCH_CACHE_DIR`| | Path to a cache directory containing launch layers +| `` | | `/cnb/lifecycle/launcher` | Path to the `launcher` executable +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_LOG_LEVEL` ` | `info` | Log Level +| `` | `CNB_PROCESS_TYPE` | | Default process type to set in the exported image +| ``| `CNB_PROCESS_TYPE` | `./project-metadata.toml` | Path to a project metadata file (see [`project-metadata.toml` (TOML)](#project-metadata.toml) +| `` | `CNB_RUN_IMAGE` | resolved from `` | Run image reference +| `` | `CNB_STACK_PATH` | `/cnb/stack.toml` | Path to stack file (see [`stack.toml` (TOML)](#stack.toml-(toml)) +| `` | `CNB_USER_ID` | | UID of the stack `User` +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml` (TOML)](#metdata.toml-(toml)) + +- At least one `` must be provided +- Each `` MUST be a valid tag reference +- **If** `` is `false` and more than one `` is provided they MUST refer to the same registry +- **If** `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of `stack` + +| Output | Description +|--------------------|---------------------------------------------- +| `[exit status]` | Success (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Exported app image (see [Buildpack Interface Specfication](buildpack.md) + +- The lifecycle SHALL write the same app image to each `` tag +- The app image: + - MUST be an extension of the `` + - All run-image layers SHALL be preserved + - All run-image config values SHALL be preserved unless this conflict with another requirement + - MUST contain all buildpack-provided launch layers as determined by the [Buildpack Interface Specfication](buildpack.md) + - MUST contain one or more app layers as determined by the [Buildpack Interface Specfication](buildpack.md) + - MUST contain a layer that includes `` + - MUST contain a layer that includes `/config/metadata.toml` + - MUST have `ENTRYPOINT=` + - MUST contain the following `Env` entries + - `"CNB_LAYERS_DIR="` + - `"CNB_APP_DIR="` + - MUST contain the following `Env` entry, if `` is set + - `"CNB_PROCESS_TYPE="` + - MUST contain the following labels + - `io.buildpacks.lifecycle.metadata`: see [lifecycle metadata label (JSON)](#lifecycle-metadata-label-(json)) + - `io.buildpacks.project.metadata`: the value of which SHALL be the json representation `` + - `io.buildpacks.build.metadata`: see [build metadata (JSON)](#build-metadata-label-(json)) +- To ensure [build reproducibility](#build-reproducibility), the lifecycle: + - SHOULD set the modification time of all files in newly created layers to a constant value + - SHOULD set the `created` time in image config to a constant value + +If a cache is provided the lifecycle: +- SHALL write the contents of all cached layers to the cache +- SHALL record the diffID and layer content metadata of all cached layers in the cache + +#### `creator` +The platform MUST execute `creator` in the **build environment** + +Usage: +``` +/cnb/lifecycle/creator \ + [-app ] \ + [-cache-dir ] \ + [-cache-image ] \ + [-daemon] \ # sets + [-gid ] \ + [-launch-cache ] \ + [-launcher ] \ + [-layers ] \ + [-log-level ] \ + [-order ] \ + [-platform ] \ + [-previous-image ] \ + [-process-type ] \ + [-project-metadata ] \ + [-run-image ] \ + [-skip-restore ] \ + [-stack ] \ + [-tag ...] \ + [-uid ] \ + +``` + +Running `creator` SHALL be equivalent to running `detector`, `analzyer`, `restorer`, `builder` and `exporter` in order with identical inputs where they are accepted, with the following exceptions. + +| Input | Environment Variable| Default Value| Description +|-------------------|---------------------|--------------|---------------------- +| ``| `CNB_PREVIOUS_IMAGE`| `` | Image reference to be analyzed (usually the result of the previous build) +| `` | `CNB_SKIP_RESTORE` | `false` | Do not write layer metadata or restore cached layers +| `...` | | | Additional tag to apply to exported image + +- **If** `` is `true` the `creator` SHALL skip layer analysis and skip the entire Restore phase. +- **If** the platform provides one or more `` inputs they SHALL be treated as additional `` inputs to the `exporter` + +Outputs produced by `creator` are identical to those produced by `exporter`. + +#### `rebaser` +Usage: +``` +/cnb/lifecycle/rebaser \ + [-daemon] \ # sets + [-gid ] \ + [-log-level ] \ + [-run-image | -image ] \ # -image is Deprecated + [-uid ] \ + [...] +``` + +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|------------------------|--------------------------------------- +| `` | `CNB_USE_DAEMON` | `false` | Export image to docker daemon +| `` | `CNB_GROUP_ID` | | Primary GID of the stack `User` +| `` | | | App image to rebase +| `` | `CNB_LOG_LEVEL` | `info` | Log Level +| `` | `CNB_RUN_IMAGE` | derived from `` | Run image reference +| `` | `CNB_USER_ID` | | UID of the stack `User` + +- At least one `` must be provided +- Each `` MUST be a valid tag reference +- **If** `` is `false` and more than one `` is provided they MUST refer to the same registry +- **If** `` is not provided by the platform the value will be [resolved](#run-image-resolution) from the contents of the `stack` key in the `io.buildpacks.lifecycle.metdata` label on ``. + +| Output | Description +|--------------------|---------------------------------------------- +| [exit status] | Pass (0), or error (1+) +| `/dev/stdout` | Logs (info) +| `/dev/stderr` | Logs (warnings, errors) +| `` | Rebased app image (see [Buildpack Interface Specfication](buildpack.md) + +- The lifecycle SHALL write the same app image to each `` tag +- The rebased app image SHALL be identical to ``, with the following modifications: + - Run image layers SHALL be defined as Layers in `` up to and including the layer with diff ID matching the value of `run-image.top-layer` from the `io.buildpacks.lifecycle.metadata` label + - Run image layers SHALL be replaced with the layers from the new `` + - The value of `io.buildpacks.lifecycle.metadata` SHALL be modified as follows + - `run-image.reference` SHALL uniquely identify `` + - `run-image.top-layer` SHALL be set to the uncompressed digest of the top layer in `` +- To ensure [build reproducibility](#build-reproducibility), the lifecycle: + - Set the `created` time in image config to a constant + +#### `launcher` +Usage: +``` +/cnb/lifecycle/launcher [--] [ ...] +``` + +| Input | Environment Variable | Default Value | Description +|---------------------|-----------------------|----------------|--------------------------------------- +| `` | `CNB_APP_DIR` | `/workspace` | Path to application directory +| `` | `CNB_LAYERS_DIR` | `/layers` | Path to layer directory +| `` | `CNB_PROCESS_TYPE` | `web` | `type` of process to launch +| `` | | | Process execution strategy +| `` | | | Command to execute +| `` | | | Arguments to command +| `/config/metadata.toml` | | | Build metadata (see [`metadata.toml`](#metdata.toml-(toml)) +| `///` | | | Launch Layers + +A command (``), arguments to that command (``), and an execution strategy (``) comprise a process definition. Processes MAY be buildpack-defined or user-defined. + +The launcher: +- MUST [select](#process-selection) a single process +- MUST construct the process execution environment as described in [Launch Environment](#launch-environment) +- MUST execute the process as specified in the [Buildpack Interface Specfication](buildpack.md) +- SHOULD replace the lifecycle with the process in memory without forking it. + +##### Process Selection +- **If** zero positional arguments are provided to the **launcher**, the lifecycle + - MUST select the process with `type` equal to `` from `/config/metadata.toml` +- **Else** + - **If** `$1` is `--` + - `` SHALL be `true` + - `` SHALL be `$2` + - `` SHALL be `${@3:}` + - **Else** + - `` SHALL be `false` + - `` SHALL be `$1` + - `` SHALL be `${@2:}` + +### Run Image Resolution + +Given stack metadata containing `run-image.image` and a set of `run-image.mirrors`. The `` for a given `` shall be resolved as follows: +- **If** any of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, this value will become the `` +- **If** none of `run-image.image` or `run-image.mirrors` has a registry matching that of ``, `` will become the `` + +### Registry Authentication + +The platform MAY set `CNB_REGISTRY_AUTH` in the lifecycle execution environment, where value of `CNB_REGISTRY_AUTH` MUST be valid JSON object and MAY contain any number of `` to `` mappings. +If `CNB_REGISTRY_AUTH` is set and `` matches the registry of an image reference, the lifecycle SHOULD set the value of the `Authorization` HTTP header to `` when attempting to read or write the image located at the given reference. + +If `CNB_REGISTRY_AUTH` is unset and a docker [config.json](https://docs.docker.com/engine/reference/commandline/cli/#configjson-properties) file is present, the lifecycle SHOULD use the contents of this file to authenticate with any matching registry. +The lifecycle SHOULD adhere to established docker conventions when checking for the existence of or interpreting the contents of a `config.json` file. + +The lifecycle MAY provide other mechanisms by which a platform can supply registry credentials. + +The lifecycle MUST attempt to authenticate anonymously if no matching credentials are found. + ## Buildpacks ### Buildpacks Directory Layout @@ -153,30 +625,122 @@ A more thorough explanation is provided in the [Buildpack Interface Specificatio ## Additional Guidance ### Environment +#### Buildpack Environment +##### Stack-Provided Variables +The following variables SHOULD be set in the lifecycle execution environment and SHALL be directly inherited by the buildpack without modification: +| Env Variable | Description +|-----------------|-------------------------------------- +| `CNB_STACK_ID` | Chosen stack ID +| `HOME` | Current user's home directory + +The following variables SHOULD be set in the lifecycle execution environment and MAY be modified by prior buildpacks before they are provided to a given buildpack: + +| Env Variable | Layer Path | Contents +|-------------------|--------------|------------------ +| `PATH` | `/bin` | binaries +| `LD_LIBRARY_PATH` | `/lib` | shared libraries +| `LIBRARY_PATH` | `/lib` | static libraries +| `CPATH` | `/include` | header files +| `PKG_CONFIG_PATH` | `/pkgconfig` | pc files + +The platform SHOULD NOT assume any other stack-provided environment variables are inherited by the buildpack. -User-provided environment variables intended for build and launch SHOULD NOT come from the same list. -The end-user SHOULD be encouraged to define them separately. -The platform MAY determine the initial environment of the build phase, detection phase, and launch. -The lifecycle MUST NOT assume that all platforms provide an identical environment. +##### User-Provided Variables +User-provided environment variables MUST be supplied by the platform as files in the `/env/` directory. +Each file SHALL define a single environment variable, where the file name defines the key and the file contents define the value. -### Run Image Rebasing +User-provided environment variables MAY be modified by prior buildpacks before they are provided to a given buildpack. -Run image rebasing allows for fast stack updates for already-exported OCI images with minimal data transfer when those images are stored on a Docker registry. -When a new stack version with the same stack ID is available, the app layers and launch layers SHOULD be rebased on the new run image by updating the image's configuration to point at the new run image. -Once the new run image is present on the registry, filesystem layers SHOULD NOT be uploaded or downloaded. +The platform SHOULD NOT set user-provided environment variables directly in the lifecycle execution environment. -The new run image MUST have an identical stack ID and MUST include the exact same set of mixins. +#### Launch Environment +User-provided modifications to the process execution environment SHOULD be set directly in the lifecycle execution environment. -![Launch](img/launch.svg) +The process SHALL inherit both stack-provided and user-provided variables from the lifecycle execution environment with the following exceptions: +* `CNB_APP_DIR`, `CNB_LAYERS_DIR` and `CNB_PROCESS_TYPE` SHALL NOT be set in the process execution environment. +* The lifecycle SHALL apply buildpack-provided modifications to the environment as outlined in the [Buildpack Interface Specification](buildpack.md). ### Caching -Each platform SHOULD implement caching so as to appropriately optimize performance. +If caching is enabled the platform is responsible for providing the lifecycle with access to the correct cache. +Whenever possible, the platform SHOULD provide the same cache to each rebuild of a given app image. Cache locality and availability MAY vary between platforms. +### Build Reproducibility +When given identical inputs all build and rebase operations: + - SHOULD produce app images with identical imageIDs + - **If** exporting directly to a registry + - SHOULD produce app images with identical manifest digests + - MAY output other non-reproducible artifacts + +To achieve reproducibility the lifecycle SHOULD set the following to a constant, rather than an accurate value: +- file modification times in generated layers +- image creation time + +Because compressions algorithms and manifest whitespace affect the image digest, an app image exported to the docker daemon and subsequently pushed to a registry MAY have a different digest than an app image exported directly to a registry by the lifecycle, even when all other inputs are held constant. + +If buildpacks do not generate layer contents or layer metadata reproducibly, builds MAY NOT be reproducibile even when identical source code and buildpacks are provided to the lifecycle. + +All app image labels SHOULD contain only reproducible values. + +For more information on build reproducibility see [https://reproducible-builds.org/](https://reproducible-builds.org/) + ## Data Format -### order.toml (TOML) +### Files + +#### `analyzed.toml` (TOML) + +```toml +[image] + reference = "" + +[metadata] +# layer metadata +``` + +Where: +- `image.reference` MUST be either a digest reference to an image in a docker registry or the ID of an image in a docker daemon +- `metadata` MUST be the TOML representation of the layer [metadata label](#layer-metadata-label-json) + +#### `group.toml` (TOML) + +```toml +group = [ + { id = "", version = "" } +] +``` + +Where: + +- Both `id` and `version` MUST be present for each buildpack object in a group. + +#### `metadata.toml` (TOML) +```toml +[[buildpacks]] +id = "" +version = "" +optional = false + +[[processes]] +type = "" +command = "" +args = [""] +direct = false + +[[slices]] +paths = [""] + +[bom] +``` + +Where: +- Both `id` and `version` MUST be present for each buildpack. +- `processes` contains the complete set of processes contributed by all buildpacks +- `processes` contains the complete set of slice defined by all buildpacks +- `bom` contains the Bill of Materials + +#### `order.toml` (TOML) ```toml [[order]] @@ -191,14 +755,186 @@ Where: - Both `id` and `version` MUST be present for each buildpack object in a group. - The value of `optional` MUST default to false if not specified. -### group.toml (TOML) +#### `plan.toml` (TOML) +```toml +[[entries]] + + [[entries.providers]] + id = "" + version = "buildpack Version" + + [[entries.requires]] + name = "" + [entries.requires.metadata] + # arbitrary data describing the required dependency +``` +Where: +- `entries` MAY be empty +- Each entry: + - MUST contain at least one buildpack in `providers` + - MUST contain at least one dependency requirement in `requires` + - MUST exclusively contain dependency requirements with the same `` + +#### `project-metadata.toml` (TOML) ```toml -group = [ - { id = "", version = "" } -] +[source] +type = "" + +[source.version] +# arbitrary data + +[source.metadata] +# arbitrary data ``` Where: +- All values are optional +- `type`, if present, SHOULD contain the type of location where the provided app source is stored (e.g `git`, `s3`) +- `version`, if present, SHOULD contain data uniquely identifying the particular version of the provided source +- `metadata` MAY contain additional arbitrary data about the provided source -- Both `id` and `version` MUST be present for each buildpack object in a group. +#### `stack.toml` (TOML) + +```toml +[run-image] + image = "" + mirrors = ["", ""] +``` + +Where: +- `run-image.image` MAY be a reference to a run image in a docker registry +- `run-image.mirrors` MUST NOT be present if `run-image.image` is not present +- `run-image.mirrors` MAY contain one or more tag references to run images in docker registries +- All `run-image.mirrors`: + - SHOULD reference an image with ID identical to that of `run-image.image` +- `run-image.image` and `run-image.mirrors.[]` SHOULD each refer to a unique registry + +### Labels + +#### `io.buildpacks.build.metadata` (JSON) + +```javascript +{ + "processes": [ + { + "type": "", + "command": "", + "args": [ + "" + ], + "direct": false + } + ], + "buildpacks": [ + { + "id": "", + "version": "" + } + ], + "bom": [ + { + "name": "", + "version": "", + "metadata": { + // arbitrary buildpack provided metadata + }, + "buildpack": { + "id": "", + "version": "" + } + }, + ], + "launcher": { + "version": "", + "source": { + "git": { + "repository": "", + "commit": "" + } + } + } +} +``` +Where: +- `processes` MUST contain all buildpack contributed processes +- `buildpacks` MUST contain the detected group +- `bom` MUST contain the Bill of Materials +- `launcher.version` SHOULD contain the version of the `launcher` binary included in the app +- `luancher.source.git.repository` SHOULD contain the git repository containing the `launcher` source code +- `luancher.source.git.commit` SHOULD contain the git commit from which the given `launcher` was built + +#### `io.buildpacks.lifecycle.metadata` (JSON) + +```javascript +{ + "app": [ + {"sha": ""} + ], + "config": { + "sha": "" + }, + "launcher": { + "sha": "" + }, + "buildpacks": [ + { + "key": "", + "version": "", + "layers": { + "": { + "sha": "", + "data": {}, + "build": false, + "launch": false, + "cache": false + } + } + } + ], + "runImage": { + "topLayer": "", + "reference": "" + }, + "stack": { + "runImage": { + "image": "cnbs/sample-stack-run:bionic" + } + } +} +``` +Where: +- `app` MUST contain one entry per app slice layer where + - `sha` MUST contain the digest of the uncompressed layer +- `config.sha` MUST the digest of the uncompressed layer containing launcher config +- `launcher.sha` MUST the digest of the uncompressed layer containing the launcher binary +- `buildpacks` MUST contain one entry per buildpack that participated in the build where + - `key` is required and MUST contain the buildpack ID + - `version` is required and MUST contain the buidpack Version + - `layers` is required and MUST contain one entry per launch layer contributed by the given buildpack. + - For each entry in `layers`: + - The key MUST be the name of the layer + - The value MUST contain JSON representation of the `layer.toml` with an additional `sha` key, containing the digest of the uncompressed layer + - The value MUST contain an additional `sha` key, containing the digest of the uncompressed layer +- `run-image.topLayer` must contain the uncompressed digest of the top layer of the run-image +- `run-image.reference` MUST uniquely identify the run image. It MAY contain one of the following + - An image ID (the digest of the uncompressed config blob) + - A digest reference to a manifest stored in an OCI image registry +- `stack` MUST contain the json representation of `stack.toml` + +#### `io.buildpacks.project.metadata` (JSON) + +```javascript +{ + "source": { + "type": "