Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A new task to generate labels #1443

Merged
merged 1 commit into from
Oct 11, 2024
Merged

A new task to generate labels #1443

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pipelines/docker-build-multi-platform-oci-ta/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|IMAGE| Reference of the image buildah will produce.| None| '$(params.output-image)'|
|IMAGE_APPEND_PLATFORM| Whether to append a sanitized platform architecture on the IMAGE tag| false| 'true'|
|IMAGE_EXPIRES_AFTER| Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.| | '$(params.image-expires-after)'|
|LABELS| Additional key=value labels that should be applied to the image| []| |
|PLATFORM| The platform to build on| None| |
|PREFETCH_INPUT| In case it is not empty, the prefetched content should be made available to the build.| | '$(params.prefetch-input)'|
|SKIP_UNUSED_STAGES| Whether to skip stages in Containerfile that seem unused by subsequent stages| true| |
Expand Down
1 change: 1 addition & 0 deletions pipelines/docker-build-oci-ta/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
|IMAGE| Reference of the image buildah will produce.| None| '$(params.output-image)'|
|IMAGE_EXPIRES_AFTER| Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.| | '$(params.image-expires-after)'|
|LABELS| Additional key=value labels that should be applied to the image| []| |
|PREFETCH_INPUT| In case it is not empty, the prefetched content should be made available to the build.| | '$(params.prefetch-input)'|
|SKIP_UNUSED_STAGES| Whether to skip stages in Containerfile that seem unused by subsequent stages| true| |
|SOURCE_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the application source code.| None| '$(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)'|
Expand Down
1 change: 1 addition & 0 deletions pipelines/docker-build/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
|IMAGE| Reference of the image buildah will produce.| None| '$(params.output-image)'|
|IMAGE_EXPIRES_AFTER| Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.| | '$(params.image-expires-after)'|
|LABELS| Additional key=value labels that should be applied to the image| []| |
|PREFETCH_INPUT| In case it is not empty, the prefetched content should be made available to the build.| | '$(params.prefetch-input)'|
|SKIP_UNUSED_STAGES| Whether to skip stages in Containerfile that seem unused by subsequent stages| true| |
|SQUASH| Squash all new and previous layers added as a part of this build, as per --squash| false| |
Expand Down
1 change: 1 addition & 0 deletions pipelines/fbc-builder/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
|IMAGE| Reference of the image buildah will produce.| None| '$(params.output-image)'|
|IMAGE_EXPIRES_AFTER| Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.| | '$(params.image-expires-after)'|
|LABELS| Additional key=value labels that should be applied to the image| []| |
|PREFETCH_INPUT| In case it is not empty, the prefetched content should be made available to the build.| | |
|SKIP_UNUSED_STAGES| Whether to skip stages in Containerfile that seem unused by subsequent stages| true| |
|SQUASH| Squash all new and previous layers added as a part of this build, as per --squash| false| |
Expand Down
1 change: 1 addition & 0 deletions task/buildah-oci-ta/0.2/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ When prefetch-dependencies task was activated it is using its artifacts to run b
|HERMETIC|Determines if build will be executed without network access.|false|false|
|IMAGE|Reference of the image buildah will produce.||true|
|IMAGE_EXPIRES_AFTER|Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.|""|false|
|LABELS|Additional key=value labels that should be applied to the image|[]|false|
|PREFETCH_INPUT|In case it is not empty, the prefetched content should be made available to the build.|""|false|
|SKIP_UNUSED_STAGES|Whether to skip stages in Containerfile that seem unused by subsequent stages|true|false|
|SOURCE_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the application source code.||true|
Expand Down
50 changes: 42 additions & 8 deletions task/buildah-oci-ta/0.2/buildah-oci-ta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ spec:
hours, days, and weeks, respectively.
type: string
default: ""
- name: LABELS
description: Additional key=value labels that should be applied to the
image
type: array
default: []
- name: PREFETCH_INPUT
description: In case it is not empty, the prefetched content should
be made available to the build.
Expand Down Expand Up @@ -226,7 +231,10 @@ spec:
- name: build
image: quay.io/konflux-ci/buildah-task:latest@sha256:860a239c5f25376a435a514ae6d53a5c75b1fa492461d17774e9b7cb32d1e275
args:
- --build-args
- $(params.BUILD_ARGS[*])
- --labels
- $(params.LABELS[*])
workingDir: /var/workdir
volumeMounts:
- mountPath: /var/lib/containers
Expand Down Expand Up @@ -302,11 +310,34 @@ spec:
sed -e '/^#/d' -e '/^\s*$/d' "${SOURCE_CODE_DIR}/${BUILD_ARGS_FILE}"
)
fi
# Append BUILD_ARGS
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
build_args+=("$@")

LABELS=()
# Split `args` into two sets of arguments.
while [[ $# -gt 0 ]]; do
case $1 in
--build-args)
shift
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
while [[ $# -gt 0 && $1 != --* ]]; do
build_args+=("$1")
shift
done
;;
--labels)
shift
while [[ $# -gt 0 && $1 != --* ]]; do
LABELS+=("--label" "$1")
shift
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's just the TA task generation doing some formatting ¯\_(ツ)_/¯

done
;;
*)
echo "unexpected argument: $1" >&2
exit 2
;;
esac
done

BUILD_ARG_FLAGS=()
for build_arg in "${build_args[@]}"; do
Expand Down Expand Up @@ -380,13 +411,16 @@ spec:
VOLUME_MOUNTS="${VOLUME_MOUNTS} --volume ${mount_point}:${YUM_REPOS_D_TARGET}"
fi

LABELS=(
DEFAULT_LABELS=(
"--label" "build-date=$(date -u +'%Y-%m-%dT%H:%M:%S')"
"--label" "architecture=$(uname -m)"
"--label" "vcs-type=git"
)
[ -n "$COMMIT_SHA" ] && LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")
[ -n "$COMMIT_SHA" ] && DEFAULT_LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && DEFAULT_LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")

# Concatenate defaults and explicit labels. If a label appears twice, the last one wins.
LABELS=("${DEFAULT_LABELS[@]}" "${LABELS[@]}")

ACTIVATION_KEY_PATH="/activation-key"
ENTITLEMENT_PATH="/entitlement"
Expand Down
49 changes: 41 additions & 8 deletions task/buildah-remote-oci-ta/0.2/buildah-remote-oci-ta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@ spec:
respectively.
name: IMAGE_EXPIRES_AFTER
type: string
- default: []
description: Additional key=value labels that should be applied to the image
name: LABELS
type: array
- default: ""
description: In case it is not empty, the prefetched content should be made available
to the build.
Expand Down Expand Up @@ -212,7 +216,10 @@ spec:
image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:e0e457b6af10e44ff6b90208a9e69adc863a865e1c062c4cb84bf3846037d74d
name: use-trusted-artifact
- args:
- --build-args
- $(params.BUILD_ARGS[*])
- --labels
- $(params.LABELS[*])
computeResources:
limits:
cpu: "4"
Expand Down Expand Up @@ -337,11 +344,34 @@ spec:
sed -e '/^#/d' -e '/^\s*$/d' "${SOURCE_CODE_DIR}/${BUILD_ARGS_FILE}"
)
fi
# Append BUILD_ARGS
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
build_args+=("$@")

LABELS=()
# Split `args` into two sets of arguments.
while [[ $# -gt 0 ]]; do
case $1 in
--build-args)
shift
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
while [[ $# -gt 0 && $1 != --* ]]; do
build_args+=("$1")
shift
done
;;
--labels)
shift
while [[ $# -gt 0 && $1 != --* ]]; do
LABELS+=("--label" "$1")
shift
done
;;
*)
echo "unexpected argument: $1" >&2
exit 2
;;
esac
done

BUILD_ARG_FLAGS=()
for build_arg in "${build_args[@]}"; do
Expand Down Expand Up @@ -415,13 +445,16 @@ spec:
VOLUME_MOUNTS="${VOLUME_MOUNTS} --volume ${mount_point}:${YUM_REPOS_D_TARGET}"
fi

LABELS=(
DEFAULT_LABELS=(
"--label" "build-date=$(date -u +'%Y-%m-%dT%H:%M:%S')"
"--label" "architecture=$(uname -m)"
"--label" "vcs-type=git"
)
[ -n "$COMMIT_SHA" ] && LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")
[ -n "$COMMIT_SHA" ] && DEFAULT_LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && DEFAULT_LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")

# Concatenate defaults and explicit labels. If a label appears twice, the last one wins.
LABELS=("${DEFAULT_LABELS[@]}" "${LABELS[@]}")

ACTIVATION_KEY_PATH="/activation-key"
ENTITLEMENT_PATH="/entitlement"
Expand Down
43 changes: 35 additions & 8 deletions task/buildah-remote/0.2/buildah-remote.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ spec:
stages
name: SKIP_UNUSED_STAGES
type: string
- default: []
description: Additional key=value labels that should be applied to the image
name: LABELS
type: array
- description: The platform to build on
name: PLATFORM
type: string
Expand Down Expand Up @@ -194,7 +198,10 @@ spec:
name: shared
steps:
- args:
- --build-args
- $(params.BUILD_ARGS[*])
- --labels
- $(params.LABELS[*])
computeResources:
limits:
cpu: "4"
Expand Down Expand Up @@ -319,11 +326,28 @@ spec:
sed -e '/^#/d' -e '/^\s*$/d' "${SOURCE_CODE_DIR}/${BUILD_ARGS_FILE}"
)
fi
# Append BUILD_ARGS
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
build_args+=("$@")

LABELS=()
# Split `args` into two sets of arguments.
while [[ $# -gt 0 ]]; do
case $1 in
--build-args)
shift
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
while [[ $# -gt 0 && $1 != --* ]]; do build_args+=("$1"); shift; done
;;
--labels)
shift
while [[ $# -gt 0 && $1 != --* ]]; do LABELS+=("--label" "$1"); shift; done
;;
*)
echo "unexpected argument: $1" >&2
exit 2
;;
esac
done

BUILD_ARG_FLAGS=()
for build_arg in "${build_args[@]}"; do
Expand Down Expand Up @@ -397,13 +421,16 @@ spec:
VOLUME_MOUNTS="${VOLUME_MOUNTS} --volume ${mount_point}:${YUM_REPOS_D_TARGET}"
fi

LABELS=(
DEFAULT_LABELS=(
"--label" "build-date=$(date -u +'%Y-%m-%dT%H:%M:%S')"
"--label" "architecture=$(uname -m)"
"--label" "vcs-type=git"
)
[ -n "$COMMIT_SHA" ] && LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")
[ -n "$COMMIT_SHA" ] && DEFAULT_LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && DEFAULT_LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")

# Concatenate defaults and explicit labels. If a label appears twice, the last one wins.
LABELS=("${DEFAULT_LABELS[@]}" "${LABELS[@]}")

ACTIVATION_KEY_PATH="/activation-key"
ENTITLEMENT_PATH="/entitlement"
Expand Down
44 changes: 36 additions & 8 deletions task/buildah/0.2/buildah.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,10 @@ spec:
description: Whether to skip stages in Containerfile that seem unused by subsequent stages
type: string
default: "true"
- name: LABELS
description: Additional key=value labels that should be applied to the image
type: array
default: []

results:
- description: Digest of the image just built
Expand Down Expand Up @@ -179,7 +183,11 @@ spec:
- name: COMMIT_SHA
value: $(params.COMMIT_SHA)
args:
- --build-args
- $(params.BUILD_ARGS[*])
- --labels
- $(params.LABELS[*])

script: |
#!/bin/bash
set -e
Expand Down Expand Up @@ -239,11 +247,28 @@ spec:
sed -e '/^#/d' -e '/^\s*$/d' "${SOURCE_CODE_DIR}/${BUILD_ARGS_FILE}"
)
fi
# Append BUILD_ARGS
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
build_args+=("$@")

LABELS=()
# Split `args` into two sets of arguments.
Copy link
Collaborator

@mmorhun mmorhun Oct 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks complicated.
Could we have additional labels in a dedicated env var for example?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started that way @mmorhun, but if we make the labels available to the script as an env var, then I don't think I can make the param an array.

Copy link
Collaborator

@mmorhun mmorhun Oct 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will a delimiter in the env var work? Like label1=value1|label2=something else

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Label values are too free-form for delimiters to work

Ralph's original approach was to use a JSON string of an array, but that's much less user-friendly

You can't really make something like this work with a JSON string

params:
  - name: LABELS
    value:
      - $(tasks.generate-labels.results.LABELS[*])
      - foo=bar

while [[ $# -gt 0 ]]; do
case $1 in
--build-args)
shift
# Note: this may result in multiple --build-arg=KEY=value flags with the same KEY being
# passed to buildah. In that case, the *last* occurrence takes precedence. This is why
# we append BUILD_ARGS after the content of the BUILD_ARGS_FILE - they take precedence.
while [[ $# -gt 0 && $1 != --* ]]; do build_args+=("$1"); shift; done
;;
--labels)
shift
while [[ $# -gt 0 && $1 != --* ]]; do LABELS+=("--label" "$1"); shift; done
;;
*)
echo "unexpected argument: $1" >&2
exit 2
;;
esac
done

BUILD_ARG_FLAGS=()
for build_arg in "${build_args[@]}"; do
Expand Down Expand Up @@ -317,13 +342,16 @@ spec:
VOLUME_MOUNTS="${VOLUME_MOUNTS} --volume ${mount_point}:${YUM_REPOS_D_TARGET}"
fi

LABELS=(
DEFAULT_LABELS=(
"--label" "build-date=$(date -u +'%Y-%m-%dT%H:%M:%S')"
"--label" "architecture=$(uname -m)"
"--label" "vcs-type=git"
)
[ -n "$COMMIT_SHA" ] && LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")
[ -n "$COMMIT_SHA" ] && DEFAULT_LABELS+=("--label" "vcs-ref=$COMMIT_SHA")
[ -n "$IMAGE_EXPIRES_AFTER" ] && DEFAULT_LABELS+=("--label" "quay.expires-after=$IMAGE_EXPIRES_AFTER")

# Concatenate defaults and explicit labels. If a label appears twice, the last one wins.
LABELS=("${DEFAULT_LABELS[@]}" "${LABELS[@]}")

ACTIVATION_KEY_PATH="/activation-key"
ENTITLEMENT_PATH="/entitlement"
Expand Down
30 changes: 30 additions & 0 deletions task/generate-labels/0.1/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# generate-labels task

Generate labels based on templates.

Usage may look like the following.

> - name: generate-labels
> params:
> - name: label-templates
> value: ["release=$SOURCE_DATE_EPOCH", "build-date=$SOURCE_DATE"]

The following environment variables are defined for use in label-templates

* ACTUAL_DATE - a date time string containing the time this task runs, formatted +'%Y-%m-%dT%H:%M:%SZ'
* ACTUAL_DATE_EPOCH - the timestamp at the time this task runs
* SOURCE_DATE - a date time string containing the provided source timestamp, formatted +'%Y-%m-%dT%H:%M:%SZ'
* SOURCE_DATE_EPOCH - the timestamp provided as a param meant to represent the timestamp at which the source was last modified


## Parameters
|name|description|default value|required|
|---|---|---|---|
|label-templates|An array of templates that should be rendered and exposed as an array of labels||true|
|source-date-epoch|A standardised environment variable for build tools to consume in order to produce reproducible output.|""|false|

## Results
|name|description|
|---|---|
|labels|The rendered labels, rendered from the provided templates|

Loading
Loading