Attach helper script #133
Merged
Attach helper script #133
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ralphbean
approved these changes
Dec 3, 2024
arewm
force-pushed
the
attach-helper-script
branch
6 times, most recently
from
4b5271e to
6c4adc3
Compare
Signed-off-by: arewm <[email protected]>
In order to make oras attach operations easier, an attach script has been added with a modified interface. This includes: * Auto-selecting the oci auth for repository-specific tokens * Use of a common artifactType * Storing the digest of the produced manifest An additional helper script is added, get-reference-base.sh, to provide a common way to remove tags and digests from OCI object references. Signed-off-by: arewm <[email protected]>
arewm
force-pushed
the
attach-helper-script
branch
from
6c4adc3 to
902cfe9
Compare
zregvart
approved these changes
Dec 4, 2024
| # contains pairs of artifacts to attach and (optionally) paths to output the blob digest | ||
| artifacts=() | ||
| artifact_type="application/vnd.konflux-ci.attached-artifact" | ||
| # distribution_spec="v1.1-referrers-api" |
There was a problem hiding this comment.
Suggested change
| # distribution_spec="v1.1-referrers-api" |
There was a problem hiding this comment.
I intentionally left this comment in as I might want to re-enable this default in the future.
Comment on lines
+117
to
+118
| oras attach "${oras_opts[@]}" --no-tty --registry-config <(select-oci-auth ${subject}) --artifact-type "${artifact_type}" \ | ||
| "${use_distribution_spec[@]}" "${subject}" "${file_name}:${media_type}" | tail -n 1 | cut -d: -f3 > "${digest_file}" |
There was a problem hiding this comment.
Suggested change
| oras attach "${oras_opts[@]}" --no-tty --registry-config <(select-oci-auth ${subject}) --artifact-type "${artifact_type}" \ | |
| "${use_distribution_spec[@]}" "${subject}" "${file_name}:${media_type}" | tail -n 1 | cut -d: -f3 > "${digest_file}" | |
| oras attach "${oras_opts[@]}" --no-tty --registry-config <(select-oci-auth ${subject}) --artifact-type "${artifact_type}" \ | |
| "${use_distribution_spec[@]}" "${subject}" "${file_name}:${media_type}" --format go-template --template '{{.digest}}' > "${digest_file}" |
Comment on lines
+18
to
+38
| original_ref="$1" | ||
|
|
||
| # Trim off digest | ||
| repo="$(echo -n $original_ref | cut -d@ -f1)" | ||
| if [[ $(echo -n "$repo" | tr -cd ":" | wc -c | tr -d '[:space:]') == 2 ]]; then | ||
| # format is now registry:port/repository:tag | ||
| # trim off everything after the last colon | ||
| repo=${repo%:*} | ||
| elif [[ $(echo -n "$repo" | tr -cd ":" | wc -c | tr -d '[:space:]') == 1 ]]; then | ||
| # we have either a port or a tag so inspect the content after | ||
| # the colon to determine if it is a valid tag. | ||
| # https://github.com/opencontainers/distribution-spec/blob/main/spec.md | ||
| # [a-zA-Z0-9_][a-zA-Z0-9._-]{0,127} is the regex for a valid tag | ||
| # If not a valid tag, leave the colon alone. | ||
| if [[ "$(echo -n "$repo" | cut -d: -f2 | tr -d '[:space:]')" =~ ^([a-zA-Z0-9_][a-zA-Z0-9._-]{0,127})$ ]]; then | ||
| # We match a tag so trim it off | ||
| repo=$(echo -n "$repo" | cut -d: -f1) | ||
| fi | ||
| fi | ||
|
|
||
| echo -n "$repo" |
There was a problem hiding this comment.
This is how I would do it to run within bash only
Suggested change
| original_ref="$1" | |
| # Trim off digest | |
| repo="$(echo -n $original_ref | cut -d@ -f1)" | |
| if [[ $(echo -n "$repo" | tr -cd ":" | wc -c | tr -d '[:space:]') == 2 ]]; then | |
| # format is now registry:port/repository:tag | |
| # trim off everything after the last colon | |
| repo=${repo%:*} | |
| elif [[ $(echo -n "$repo" | tr -cd ":" | wc -c | tr -d '[:space:]') == 1 ]]; then | |
| # we have either a port or a tag so inspect the content after | |
| # the colon to determine if it is a valid tag. | |
| # https://github.com/opencontainers/distribution-spec/blob/main/spec.md | |
| # [a-zA-Z0-9_][a-zA-Z0-9._-]{0,127} is the regex for a valid tag | |
| # If not a valid tag, leave the colon alone. | |
| if [[ "$(echo -n "$repo" | cut -d: -f2 | tr -d '[:space:]')" =~ ^([a-zA-Z0-9_][a-zA-Z0-9._-]{0,127})$ ]]; then | |
| # We match a tag so trim it off | |
| repo=$(echo -n "$repo" | cut -d: -f1) | |
| fi | |
| fi | |
| echo -n "$repo" | |
| # 1. anything not slash or colon | |
| # 2. colon and numbers (optional) | |
| # slash | |
| # 3. anything but colon and at symbol | |
| # 4. everything else | |
| imageref_rx='^([^/:]+)(:[[:digit:]]+)?/([^:@]+)(.*)$' | |
| if [[ "$1" =~ ${imageref_rx} ]]; then | |
| host="${BASH_REMATCH[1]}" | |
| port="${BASH_REMATCH[2]}" | |
| repository="${BASH_REMATCH[3]}" | |
| echo "${host}${port}/${repository}" | |
| fi |
There was a problem hiding this comment.
([^/:]+)(:[[:digit:]]+)? is what matches the host, but isn't that is split across two of the rematches?
|
Merging this PR after approvals to unblock work in build-definitions. We can iterate on the refinement/nitpicks in future PRs. |
mmalina
added a commit
to mmalina/release-service-utils
that referenced
this pull request
Dec 12, 2024
`select-oci-auth` stopped working when the oras image ref was updated to the latest one. The `select-oci-auth` in the oras-container repo was refactored and part of the logic was moved to a new `get-reference-base` script, which it uses. So we need to add that script. Related PR: konflux-ci/oras-container#133 Signed-off-by: Martin Malina <[email protected]>
mmalina
added a commit
to konflux-ci/release-service-utils
that referenced
this pull request
Dec 12, 2024
`select-oci-auth` stopped working when the oras image ref was updated to the latest one. The `select-oci-auth` in the oras-container repo was refactored and part of the logic was moved to a new `get-reference-base` script, which it uses. So we need to add that script. Related PR: konflux-ci/oras-container#133 Signed-off-by: Martin Malina <[email protected]>
|
This broke release-service-utils. Fixed today here: konflux-ci/release-service-utils#341 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to make oras attach operations easier, an attach script has
been added with a modified interface. This includes:
An additional helper script is added, get-reference-base.sh, to provide
a common way to remove tags and digests from OCI object references.
This PR is an alternative implementation for #130. Only one of the two needs to be approved and merged.