Skip to content

Commit

Permalink
IMAGE_REF result from image building Tasks
Browse files Browse the repository at this point in the history
This helps in the reuse of the results when using matrix feature of
Tekton. Given that the concatenation of two results from matrix-spawned
Tasks is not supported, e.g.

    $(tasks.build-container-multiarch.results.IMAGE_URL[*])@$(tasks.build-container-multiarch.results.IMAGE_DIGEST[*])

will not expand correctly.

This produces the image reference in full in the `IMAGE_REF` result, so
the result from the matrix-spawned Tasks can be referenced using:

    $(tasks.build-container-multiarch.results.IMAGE_REF[*])

Reference: https://issues.redhat.com/browse/EC-654
  • Loading branch information
zregvart committed Jul 9, 2024
1 parent 3c4314e commit 2b6bbba
Show file tree
Hide file tree
Showing 24 changed files with 206 additions and 124 deletions.
1 change: 1 addition & 0 deletions task/buildah-oci-ta/0.1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ When prefetch-dependencies task was activated it is using its artifacts to run b
|---|---|
|BASE_IMAGES_DIGESTS|Digests of the base images used for build|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_REF|Image reference of the built image|
|IMAGE_URL|Image repository where the built image was pushed|
|JAVA_COMMUNITY_DEPENDENCIES|The Java dependencies that came from community sources such as Maven central.|
|SBOM_JAVA_COMPONENTS_COUNT|The counting of Java components by publisher in JSON format|
Expand Down
6 changes: 6 additions & 0 deletions task/buildah-oci-ta/0.1/buildah-oci-ta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,8 @@ spec:
description: Digests of the base images used for build
- name: IMAGE_DIGEST
description: Digest of the image just built
- name: IMAGE_REF
description: Image reference of the built image
- name: IMAGE_URL
description: Image repository where the built image was pushed
- name: JAVA_COMMUNITY_DEPENDENCIES
Expand Down Expand Up @@ -528,6 +530,10 @@ spec:
cat "/var/workdir"/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -n "$IMAGE" | tee $(results.IMAGE_URL.path)
{
echo -n "${IMAGE}@"
cat "/var/workdir/image-digest"
} >"$(results.IMAGE_REF.path)"
securityContext:
capabilities:
add:
Expand Down
45 changes: 45 additions & 0 deletions task/buildah-remote-oci-ta/0.1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# buildah-remote-oci-ta task

Buildah task builds source code into a container image and pushes the image into container registry using buildah tool.
In addition it generates a SBOM file, injects the SBOM file into final container image and pushes the SBOM file as separate image using cosign tool.
When [Java dependency rebuild](https://redhat-appstudio.github.io/docs.stonesoup.io/Documentation/main/cli/proc_enabled_java_dependencies.html) is enabled it triggers rebuilds of Java artifacts.
When prefetch-dependencies task was activated it is using its artifacts to run build in hermetic environment.

## Parameters
|name|description|default value|required|
|---|---|---|---|
|ADDITIONAL_SECRET|Name of a secret which will be made available to the build with 'buildah build --secret' at /run/secrets/$ADDITIONAL_SECRET|does-not-exist|false|
|ADD_CAPABILITIES|Comma separated list of extra capabilities to add when running 'buildah build'|""|false|
|BUILD_ARGS|Array of --build-arg values ("arg=value" strings)|[]|false|
|BUILD_ARGS_FILE|Path to a file with build arguments, see https://www.mankier.com/1/buildah-build#--build-arg-file|""|false|
|CACHI2_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.|""|false|
|COMMIT_SHA|The image is built from this commit.|""|false|
|CONTEXT|Path to the directory to use as context.|.|false|
|DOCKERFILE|Path to the Dockerfile to build.|./Dockerfile|false|
|DOCKER_AUTH|unused, should be removed in next task version|""|false|
|ENTITLEMENT_SECRET|Name of secret which contains the entitlement certificates|etc-pki-entitlement|false|
|HERMETIC|Determines if build will be executed without network access.|false|false|
|IMAGE|Reference of the image buildah will produce.||true|
|IMAGE_EXPIRES_AFTER|Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.|""|false|
|PREFETCH_INPUT|In case it is not empty, the prefetched content should be made available to the build.|""|false|
|SOURCE_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the application source code.||true|
|SQUASH|Squash all new and previous layers added as a part of this build, as per --squash|false|false|
|TARGET_STAGE|Target stage in Dockerfile to build. If not specified, the Dockerfile is processed entirely to (and including) its last stage.|""|false|
|TLSVERIFY|Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)|true|false|
|YUM_REPOS_D_FETCHED|Path in source workspace where dynamically-fetched repos are present|fetched.repos.d|false|
|YUM_REPOS_D_SRC|Path in the git repository in which yum repository files are stored|repos.d|false|
|YUM_REPOS_D_TARGET|Target path on the container in which yum repository files should be made available|/etc/yum.repos.d|false|
|caTrustConfigMapKey|The name of the key in the ConfigMap that contains the CA bundle data.|ca-bundle.crt|false|
|caTrustConfigMapName|The name of the ConfigMap to read CA bundle data from.|trusted-ca|false|
|PLATFORM|The platform to build on||true|

## Results
|name|description|
|---|---|
|BASE_IMAGES_DIGESTS|Digests of the base images used for build|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_REF|Image reference of the built image|
|IMAGE_URL|Image repository where the built image was pushed|
|JAVA_COMMUNITY_DEPENDENCIES|The Java dependencies that came from community sources such as Maven central.|
|SBOM_JAVA_COMPONENTS_COUNT|The counting of Java components by publisher in JSON format|

6 changes: 6 additions & 0 deletions task/buildah-remote-oci-ta/0.1/buildah-remote-oci-ta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,8 @@ spec:
name: BASE_IMAGES_DIGESTS
- description: Digest of the image just built
name: IMAGE_DIGEST
- description: Image reference of the built image
name: IMAGE_REF
- description: Image repository where the built image was pushed
name: IMAGE_URL
- description: The Java dependencies that came from community sources such as Maven
Expand Down Expand Up @@ -598,6 +600,10 @@ spec:
cat "/var/workdir"/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -n "$IMAGE" | tee $(results.IMAGE_URL.path)
{
echo -n "${IMAGE}@"
cat "/var/workdir/image-digest"
} >"$(results.IMAGE_REF.path)"
securityContext:
capabilities:
add:
Expand Down
49 changes: 30 additions & 19 deletions task/buildah-remote/0.1/README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,43 @@
# buildah-remote task

This task is programmatically generated from the buildah task to keep it in sync, it should not be manually modified.

Buildah task builds source code into a container image and pushes the image into container registry using buildah tool, however while the standard buildah task is run directly on the cluster, this task is run on a remote host. This must be used in combination with the [Multi Arch Controller](https://github.com/redhat-appstudio/multi-arch-controller) which provides the credentials and host name to use to perform the build.

This task has an additional `PLATFORM` param that is used by the Multi Arch Controller to decide which host should perform the build.


Buildah task builds source code into a container image and pushes the image into container registry using buildah tool.
In addition it generates a SBOM file, injects the SBOM file into final container image and pushes the SBOM file as separate image using cosign tool.
When [Java dependency rebuild](https://redhat-appstudio.github.io/docs.stonesoup.io/Documentation/main/cli/proc_enabled_java_dependencies.html) is enabled it triggers rebuilds of Java artifacts.
When prefetch-dependencies task was activated it is using its artifacts to run build in hermetic environment.

## Parameters
| name | description |default value|required|
|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|---|---|
| PLATFORM | The target platform. ||true|
| IMAGE | Reference of the image buildah will produce. ||true|
| BUILDER_IMAGE | The location of the buildah builder image. |registry.access.redhat.com/ubi9/buildah:9.0.0-19@sha256:c8b1d312815452964885680fc5bc8d99b3bfe9b6961228c71a09c72ca8e915eb|false|
| DOCKERFILE | Path to the Dockerfile to build. |./Dockerfile|false|
| CONTEXT | Path to the directory to use as context. |.|false|
| TLSVERIFY | Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry) |true|false|
| DOCKER_AUTH | unused, should be removed in next task version |""|false|
| HERMETIC | Determines if build will be executed without network access. |false|false|
| PREFETCH_INPUT | In case it is not empty, the prefetched content should be made available to the build. |""|false|
| IMAGE_EXPIRES_AFTER | Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively. |""|false|
|name|description|default value|required|
|---|---|---|---|
|IMAGE|Reference of the image buildah will produce.||true|
|BUILDER_IMAGE|Deprecated. Has no effect. Will be removed in the future.|""|false|
|DOCKERFILE|Path to the Dockerfile to build.|./Dockerfile|false|
|CONTEXT|Path to the directory to use as context.|.|false|
|TLSVERIFY|Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)|true|false|
|DOCKER_AUTH|unused, should be removed in next task version|""|false|
|HERMETIC|Determines if build will be executed without network access.|false|false|
|PREFETCH_INPUT|In case it is not empty, the prefetched content should be made available to the build.|""|false|
|IMAGE_EXPIRES_AFTER|Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.|""|false|
|COMMIT_SHA|The image is built from this commit.|""|false|
|YUM_REPOS_D_SRC|Path in the git repository in which yum repository files are stored|repos.d|false|
|YUM_REPOS_D_FETCHED|Path in source workspace where dynamically-fetched repos are present|fetched.repos.d|false|
|YUM_REPOS_D_TARGET|Target path on the container in which yum repository files should be made available|/etc/yum.repos.d|false|
|TARGET_STAGE|Target stage in Dockerfile to build. If not specified, the Dockerfile is processed entirely to (and including) its last stage.|""|false|
|ENTITLEMENT_SECRET|Name of secret which contains the entitlement certificates|etc-pki-entitlement|false|
|ADDITIONAL_SECRET|Name of a secret which will be made available to the build with 'buildah build --secret' at /run/secrets/$ADDITIONAL_SECRET|does-not-exist|false|
|BUILD_ARGS|Array of --build-arg values ("arg=value" strings)|[]|false|
|BUILD_ARGS_FILE|Path to a file with build arguments, see https://www.mankier.com/1/buildah-build#--build-arg-file|""|false|
|caTrustConfigMapName|The name of the ConfigMap to read CA bundle data from.|trusted-ca|false|
|caTrustConfigMapKey|The name of the key in the ConfigMap that contains the CA bundle data.|ca-bundle.crt|false|
|ADD_CAPABILITIES|Comma separated list of extra capabilities to add when running 'buildah build'|""|false|
|SQUASH|Squash all new and previous layers added as a part of this build, as per --squash|false|false|
|PLATFORM|The platform to build on||true|

## Results
|name|description|
|---|---|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_URL|Image repository where the built image was pushed|
|IMAGE_REF|Image reference of the built image|
|BASE_IMAGES_DIGESTS|Digests of the base images used for build|
|SBOM_JAVA_COMPONENTS_COUNT|The counting of Java components by publisher in JSON format|
|JAVA_COMMUNITY_DEPENDENCIES|The Java dependencies that came from community sources such as Maven central.|
Expand Down
6 changes: 6 additions & 0 deletions task/buildah-remote/0.1/buildah-remote.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ spec:
name: IMAGE_DIGEST
- description: Image repository where the built image was pushed
name: IMAGE_URL
- description: Image reference of the built image
name: IMAGE_REF
- description: Digests of the base images used for build
name: BASE_IMAGES_DIGESTS
- description: The counting of Java components by publisher in JSON format
Expand Down Expand Up @@ -595,6 +597,10 @@ spec:
cat "$(workspaces.source.path)"/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -n "$IMAGE" | tee $(results.IMAGE_URL.path)
{
echo -n "${IMAGE}@"
cat "$(workspaces.source.path)/image-digest"
} > "$(results.IMAGE_REF.path)"
securityContext:
capabilities:
add:
Expand Down
4 changes: 4 additions & 0 deletions task/buildah/0.1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,17 @@ When prefetch-dependencies task was activated it is using its artifacts to run b
|ADDITIONAL_SECRET|Name of a secret which will be made available to the build with 'buildah build --secret' at /run/secrets/$ADDITIONAL_SECRET|does-not-exist|false|
|BUILD_ARGS|Array of --build-arg values ("arg=value" strings)|[]|false|
|BUILD_ARGS_FILE|Path to a file with build arguments, see https://www.mankier.com/1/buildah-build#--build-arg-file|""|false|
|caTrustConfigMapName|The name of the ConfigMap to read CA bundle data from.|trusted-ca|false|
|caTrustConfigMapKey|The name of the key in the ConfigMap that contains the CA bundle data.|ca-bundle.crt|false|
|ADD_CAPABILITIES|Comma separated list of extra capabilities to add when running 'buildah build'|""|false|
|SQUASH|Squash all new and previous layers added as a part of this build, as per --squash|false|false|

## Results
|name|description|
|---|---|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_URL|Image repository where the built image was pushed|
|IMAGE_REF|Image reference of the built image|
|BASE_IMAGES_DIGESTS|Digests of the base images used for build|
|SBOM_JAVA_COMPONENTS_COUNT|The counting of Java components by publisher in JSON format|
|JAVA_COMMUNITY_DEPENDENCIES|The Java dependencies that came from community sources such as Maven central.|
Expand Down
6 changes: 6 additions & 0 deletions task/buildah/0.1/buildah.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ spec:
name: IMAGE_DIGEST
- description: Image repository where the built image was pushed
name: IMAGE_URL
- description: Image reference of the built image
name: IMAGE_REF
- description: Digests of the base images used for build
name: BASE_IMAGES_DIGESTS
- name: SBOM_JAVA_COMPONENTS_COUNT
Expand Down Expand Up @@ -497,6 +499,10 @@ spec:
cat "$(workspaces.source.path)"/image-digest | tee $(results.IMAGE_DIGEST.path)
echo -n "$IMAGE" | tee $(results.IMAGE_URL.path)
{
echo -n "${IMAGE}@"
cat "$(workspaces.source.path)/image-digest"
} > "$(results.IMAGE_REF.path)"
securityContext:
runAsUser: 0
Expand Down
1 change: 1 addition & 0 deletions task/oci-copy-oci-ta/0.1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Given a file in the user's source directory, copy content from arbitrary urls in
|name|description|
|---|---|
|IMAGE_DIGEST|Digest of the artifact just pushed|
|IMAGE_REF|Image reference of the built image|
|IMAGE_URL|Repository where the artifact was pushed|
|SBOM_BLOB_URL|Link to the SBOM blob pushed to the registry.|

3 changes: 3 additions & 0 deletions task/oci-copy-oci-ta/0.1/oci-copy-oci-ta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ spec:
results:
- name: IMAGE_DIGEST
description: Digest of the artifact just pushed
- name: IMAGE_REF
description: Image reference of the built image
- name: IMAGE_URL
description: Repository where the artifact was pushed
- name: SBOM_BLOB_URL
Expand Down Expand Up @@ -186,6 +188,7 @@ spec:
RESULTING_DIGEST=$(oras resolve --registry-config auth.json "${IMAGE}")
echo -n "$RESULTING_DIGEST" | tee "$(results.IMAGE_DIGEST.path)"
echo -n "$IMAGE" | tee "$(results.IMAGE_URL.path)"
echo -n "${IMAGE}@${IMAGE_INDEX_DIGEST}" >"$(results.IMAGE_REF.path)"
computeResources:
limits:
memory: 1Gi
Expand Down
61 changes: 7 additions & 54 deletions task/oci-copy/0.1/README.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,23 @@
# oci-copy task

Given an `oci-copy.yaml` file in the user's source directory, the `oci-copy` task will copy content from arbitrary urls into the OCI registry.

It generates a limited SBOM and pushes that into the OCI registry alongside the image.

It is not to be considered safe for general use as it cannot provide a high degree of provenance for artficats and reports them only as "general" type artifacts in the purl spec it reports in the SBOM. Use only in limited situations.

Note: the bearer token secret, if specified, will be sent to **all servers listed in the oci-copy.yaml file**.
Given a file in the user's source directory, copy content from arbitrary urls into the OCI registry.

## Parameters
|name|description|default value|required|
|---|---|---|---|
|IMAGE|Reference of the image buildah will produce.||true|
|IMAGE|Reference of the image we will push||true|
|OCI_COPY_FILE|Path to the oci copy file.|./oci-copy.yaml|false|
|BEARER_TOKEN_SECRET_NAME|Name of a secret which will be made available to the build as an Authorization header. Note, the token will be sent to all servers found in the oci-copy.yaml file. If you do not wish to send the token to all servers, different taskruns and therefore different oci artifacts must be used.|"does-not-exist"|false|

|BEARER_TOKEN_SECRET_NAME|Name of a secret which will be made available to the build as an Authorization header. Note, the token will be sent to all servers found in the oci-copy.yaml file. If you do not wish to send the token to all servers, different taskruns and therefore different oci artifacts must be used.|does-not-exist|false|

## Results
|name|description|
|---|---|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_URL|Image repository where the built image was pushed|
|IMAGE_DIGEST|Digest of the artifact just pushed|
|IMAGE_URL|Repository where the artifact was pushed|
|SBOM_BLOB_URL|Link to the SBOM blob pushed to the registry.|
|IMAGE_REF|Image reference of the built image|

## Workspaces
|name|description|optional|
|---|---|---|
|source|Workspace containing the source code to copy.|false|

## oci-copy.yaml schema
JSON schema for the `oci-copy.yaml` file.

```json
{
"type": "object",
"required": ["artifacts", "artifact_type"],
"properties": {
"artifact_type": {
"description": "Artifact type to be applied to the top-level OCI artifact, i.e. `application/x-mlmodel`",
"type": "string"
},
"artifacts": {
"type": "array",
"items": {
"type": "object",
"required": ["source", "filename", "type", "sha256sum"],
"properties": {
"source": {
"description": "URL of the artifact to copy",
"type": "string"
},
"filename": {
"description": "Filename that should be applied to the artifact in the OCI registry",
"type": "string"
},
"type": {
"description": "Media type that should be applied to the artifact in the OCI registry",
"type": "string"
},
"sha256sum": {
"description": "Digest of the artifact to be checked before copy",
"type": "string"
}
}
}
}
}
}
```
|source|Workspace containing the source artifacts to copy|false|
Loading

0 comments on commit 2b6bbba

Please sign in to comment.