Improve generation of base images SBOMs #191
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mkosiarc
force-pushed
the
parse-base-images-from-json
branch
2 times, most recently
from
307dab0 to
16c1454
Compare
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Nov 28, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile>|<full-image-reference-with-digest> The character that creates the mapping "|" was chosen arbitrarily, just because it cannot be a valid part of image reference. KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Nov 28, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile>|<full-image-reference-with-digest> The character that creates the mapping "|" was chosen arbitrarily, just because it cannot be a valid part of image reference. KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Nov 28, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile>|<full-image-reference-with-digest> The character that creates the mapping "|" was chosen arbitrarily, just because it cannot be a valid part of image reference. KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
mkosiarc
force-pushed
the
parse-base-images-from-json
branch
3 times, most recently
from
53b524a to
832c174
Compare
chmeliik
reviewed
Nov 28, 2024
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Show resolved
Hide resolved
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
tkdchen
reviewed
Nov 29, 2024
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
mkosiarc
force-pushed
the
parse-base-images-from-json
branch
2 times, most recently
from
e04ae40 to
2177875
Compare
|
switched to using empty space, instead of "|" as discussed in konflux-ci/build-definitions#1659 |
chmeliik
approved these changes
Dec 4, 2024
sbom-utility-scripts/scripts/base-images-sbom-script/app/base_images_sbom_script.py
Outdated
Show resolved
Hide resolved
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Dec 4, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile> <full-image-reference-with-digest> KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Dec 4, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile> <full-image-reference-with-digest> KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
The process how the base image SBOM is generated was slightly changed. The most important change is due to requiring different inputs. The script now requires the full json of parsed Dockerfile, so we can extract the base images better. This allows us to extract base images data even for stage aliases. Another advantage is that we are counting the stages/layers more carefully, because we have more information about which stage was from scratch or from oci-archive. KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
mkosiarc
force-pushed
the
parse-base-images-from-json
branch
from
2177875 to
16f74d5
Compare
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Dec 4, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile> <full-image-reference-with-digest> KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
tkdchen
approved these changes
Dec 9, 2024
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Dec 9, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile> <full-image-reference-with-digest> KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
mkosiarc
added a commit
to mkosiarc/build-definitions
that referenced
this pull request
Dec 9, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile> <full-image-reference-with-digest> Also, the sbom utility image has to be updated together in the same PR/commit, otherwise it would break konflux temporarily KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
github-merge-queue bot
pushed a commit
to konflux-ci/build-definitions
that referenced
this pull request
Dec 9, 2024
most functional changes are in the related PR that updates the base_images_sbom_script.py konflux-ci/build-tasks-dockerfiles#191 Here, we are just updating on how we generate the inputs for this script. We are now passing the whole parsed Dockerfile in json format to that script, which allows us to better parse/detect base images. Also, the format of the /shared/base_images_digests file was changed. Previously we could rely on the order of the image references with the digests in the file. Now we need to provide a mapping from an image reference as it was used in the Dockerfile to the full image reference with digests that was used during build and generated by buildah. The mapping is done as: <image-reference-used-in-dockerfile> <full-image-reference-with-digest> Also, the sbom utility image has to be updated together in the same PR/commit, otherwise it would break konflux temporarily KFLUXBUGS-1718 Signed-off-by: mkosiarc <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The process how the base image SBOM is generated was slightly changed.
The most important change is due to requiring different inputs. The script now requires the full json of parsed Dockerfile, so we can extract the base images better. This allows us to extract base images data even for stage aliases.
Another advantage is that we are counting
the stages/layers more carefully, because we have more information about which stage was from scratch or from oci-archive.
KFLUXBUGS-1718