metadata: Add logic to fallback to quay if mirror is unreachable #3802

praveenkumar · 2023-08-25T07:54:14Z

Since we are going to put bundles on quay and mirror where quay serve as
fallback solution. This PR makes user aware in case mirror fails to
download bundle.

INFO Getting bundle for the CRC executable
INFO Downloading bundle: /home/prkumar/.crc/cache/crc_microshift_libvirt_4.13.9_amd64.crcbundle...
INFO Unable to download bundle from mirror, falling back to quay
Getting image source signatures
Copying blob d37da0367460 done
Copying config f3021495d6 done
Writing manifest to image destination
Storing signatures
INFO Extracting the image bundle layer...
crc_microshift_libvirt_4.13.9_amd64.crcbundle:  1.46 GiB / 1.46 GiB [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
INFO Verifying the bundle signature...
INFO Uncompressing /home/prkumar/.crc/cache/crc_microshift_libvirt_4.13.9_amd64.crcbundle
crc.qcow2:  4.28 GiB / 4.28 GiB [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
oc:  141.85 MiB / 141.85 MiB [-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
Your system is correctly setup for using CRC. Use 'crc start' to start the instance

cfergeau · 2023-08-25T08:01:45Z

pkg/crc/machine/bundle/metadata.go

@@ -353,7 +353,12 @@ func Download(preset crcPreset.Preset, bundleURI string) (string, error) {
 	// bundles, their sha256sums are known and can be checked.
 	if bundleURI == constants.GetDefaultBundlePath(preset) {
 		if preset == crcPreset.OpenShift || preset == crcPreset.Microshift {
-			return DownloadDefault(preset)
+			downloadedBundlePath, err := DownloadDefault(preset)


Is there a way to differentiate "not found" from other errors?

@cfergeau looks like grab package have StatusCodeError type and can validated with IsStatusCodeError which I now used.

StatusCodeError indicates that the server response had a status code that // was not in the 200-299 range (after following any redirects).

pkg/crc/machine/bundle/metadata.go

pkg/crc/image/image.go

cfergeau · 2023-09-29T10:37:22Z

This PR needs a rebase.

This remove some of code duplication.

cfergeau

Overall looks good, a few minor comments left.

pkg/crc/image/image.go

pkg/crc/config/settings.go

cfergeau · 2023-09-29T13:25:06Z

pkg/crc/preflight/preflight.go

 	logging.Infof("Using bundle path %s", bundlePath)
-	return getPreflightChecks(experimentalFeatures, mode, bundlePath, preset)
+	return getPreflightChecks(experimentalFeatures, mode, bundlePath, preset, enableBundleQuayFallback)


Hrm, 4 arguments was already quite a few, 5 is starting to be too many args :-/

cfergeau · 2023-09-29T13:27:02Z

pkg/crc/machine/bundle/metadata.go

@@ -346,7 +348,12 @@ func Download(preset crcPreset.Preset, bundleURI string) (string, error) {
 	if bundleURI == constants.GetDefaultBundlePath(preset) {
 		switch preset {
 		case crcPreset.OpenShift, crcPreset.Microshift:
-			return downloadDefault(preset)
+			downloadedBundlePath, err := downloadDefault(preset)
+			if grab.IsStatusCodeError(errors.Unwrap(err)) && enableBundleQuayFallback {


Looks like this could be errors.Is(err, grab.StatusCodeError)

@cfergeau this is not work because as per errors.Is it doesn't call the unwrap on the target and the way we have this error it is wrapped around grabStatusCodeError so it is not able to work.

errors.Is was introduced to deal with wrapped errors. Its documentation is clear about this. https://pkg.go.dev/errors

Yes but if I take a look to grab error package the implementation around StatusCodeError is type int and if I try to use errors.Is on that it is not working until we know the statuscode. I think that is the reason it have IsStatusCodeError function.

https://go.dev/play/p/jwkn9TG9SmH

Ah right, errors.Is checks for equality, not for type :-/ errors.As checks for types, but is not as convenient as errors.Is. If we keep this code, I think we should use errors.As instead of hardcoding a single errors.Unwrap(). Hardcoding this Unwrap() will break if we move/refactor this code, and the level of wrapping is changed (eg if errors.Unwrap(errors.Unwrap()) becomes needed instead of errors.Unwrap()).

However looking at downloadDefault(), there are more situations than "the bundle file is missing/cannot be downloaded" when we want to fallback, for example if the signature is missing/invalid. IsStatusCodeError() seems to be too restrictive? The initial version where the fallback always happened when there was an error may be a better option

@cfergeau yeah that is more flexible, I can revert to that.

As part of a9c0b4c we didn't add microshift preset. This PR add microshift preset to fetch default container image when it is available to `quay.io`.

openshift-merge-robot · 2023-10-03T11:37:48Z

@praveenkumar: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/integration-crc	`f1ed74e`	link	true	`/test integration-crc`
ci/prow/e2e-crc	`f1ed74e`	link	true	`/test e2e-crc`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Since we are going to put bundles on quay and mirror where quay serve as fallback solution. This PR add a config flag `enable-bundle-quay-fallback` which allow user to download bundle from quay. ``` $ crc config set enable-bundle-quay-fallback true $ crc setup INFO Getting bundle for the CRC executable INFO Downloading bundle: /home/prkumar/.crc/cache/crc_microshift_libvirt_4.13.9_amd64.crcbundle... INFO Unable to download bundle from mirror, falling back to quay Getting image source signatures Copying blob d37da0367460 done Copying config f3021495d6 done Writing manifest to image destination Storing signatures INFO Extracting the image bundle layer... crc_microshift_libvirt_4.13.9_amd64.crcbundle: 1.46 GiB / 1.46 GiB [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% INFO Verifying the bundle signature... INFO Uncompressing /home/prkumar/.crc/cache/crc_microshift_libvirt_4.13.9_amd64.crcbundle crc.qcow2: 4.28 GiB / 4.28 GiB [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% oc: 141.85 MiB / 141.85 MiB [-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% Your system is correctly setup for using CRC. Use 'crc start' to start the instance ```

openshift-ci · 2023-10-04T14:04:11Z

@praveenkumar: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/integration-crc	`fea84f0`	link	true	`/test integration-crc`
ci/prow/e2e-crc	`fea84f0`	link	true	`/test e2e-crc`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci · 2023-10-05T10:43:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfergeau

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [cfergeau]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot requested review from anjannath and evidolob August 25, 2023 07:54

cfergeau reviewed Aug 25, 2023

View reviewed changes

cfergeau reviewed Aug 30, 2023

View reviewed changes

pkg/crc/image/image.go Outdated Show resolved Hide resolved

praveenkumar force-pushed the fallback_quay branch from f0f606b to 504ffef Compare September 4, 2023 06:19

openshift-merge-robot added the needs-rebase label Sep 5, 2023

praveenkumar force-pushed the fallback_quay branch from 504ffef to 048a2d7 Compare September 26, 2023 08:29

openshift-merge-robot removed the needs-rebase label Sep 26, 2023

praveenkumar force-pushed the fallback_quay branch from 048a2d7 to f0e1b73 Compare September 26, 2023 08:55

praveenkumar self-assigned this Sep 26, 2023

praveenkumar force-pushed the fallback_quay branch from f0e1b73 to d153865 Compare September 26, 2023 10:18

openshift-merge-robot added the needs-rebase label Sep 29, 2023

praveenkumar force-pushed the fallback_quay branch from d153865 to 39df44e Compare September 29, 2023 10:43

openshift-merge-robot removed the needs-rebase label Sep 29, 2023

Preflight: Add getPreflightChecksHelper function

5a088e9

This remove some of code duplication.

praveenkumar force-pushed the fallback_quay branch from 39df44e to 2833e21 Compare September 29, 2023 10:49

cfergeau reviewed Sep 29, 2023

View reviewed changes

image: Add microshift preset to pull container image

e48506a

As part of a9c0b4c we didn't add microshift preset. This PR add microshift preset to fetch default container image when it is available to `quay.io`.

praveenkumar force-pushed the fallback_quay branch from 2833e21 to f1ed74e Compare October 3, 2023 09:12

praveenkumar force-pushed the fallback_quay branch from f1ed74e to fea84f0 Compare October 4, 2023 11:17

cfergeau approved these changes Oct 5, 2023

View reviewed changes

openshift-ci bot assigned cfergeau Oct 5, 2023

openshift-ci bot added the lgtm label Oct 5, 2023

openshift-ci bot added the approved label Oct 5, 2023

praveenkumar merged commit 2875a74 into crc-org:main Oct 6, 2023
9 of 13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

metadata: Add logic to fallback to quay if mirror is unreachable #3802

metadata: Add logic to fallback to quay if mirror is unreachable #3802

praveenkumar commented Aug 25, 2023

cfergeau Aug 25, 2023

praveenkumar Sep 4, 2023

cfergeau commented Sep 29, 2023

cfergeau left a comment

cfergeau Sep 29, 2023

cfergeau Sep 29, 2023

praveenkumar Oct 3, 2023

cfergeau Oct 3, 2023

praveenkumar Oct 4, 2023

cfergeau Oct 4, 2023 •

edited

Loading

praveenkumar Oct 4, 2023

praveenkumar Oct 5, 2023

openshift-merge-robot commented Oct 3, 2023

openshift-ci bot commented Oct 4, 2023

openshift-ci bot commented Oct 5, 2023

metadata: Add logic to fallback to quay if mirror is unreachable #3802

metadata: Add logic to fallback to quay if mirror is unreachable #3802

Conversation

praveenkumar commented Aug 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfergeau commented Sep 29, 2023

cfergeau left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfergeau Oct 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-merge-robot commented Oct 3, 2023

openshift-ci bot commented Oct 4, 2023

openshift-ci bot commented Oct 5, 2023

cfergeau Oct 4, 2023 •

edited

Loading