CRIO: add docker.io to the unqualified-search-registries

[medyagh]

CRIO fails when you deploy an image without docker.io prefix

for example
$ kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0
deployment.apps/hello-minikube created

default       hello-minikube-5c898d8489-8gh5t    0/1     ImageInspectError   0          15s

Failed to inspect image "kicbase/echo-server:1.0": rpc error: code = Unknown desc = short-name "kicbase/echo-server:1.0" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"

after adding this line

unqualified-search-registries = ["docker.io"] 

to the
sudo cat /etc/containers/registries.conf

it works

docker@minikube:~$ sudo cat /etc/containers/registries.conf
# For more information on this configuration file, see containers-registries.conf(5).
#
# NOTE: RISK OF USING UNQUALIFIED IMAGE NAMES
# We recommend always using fully qualified image names including the registry
# server (full dns name), namespace, image name, and tag
# (e.g., registry.redhat.io/ubi8/ubi:latest). Pulling by digest (i.e.,
# quay.io/repository/name@digest) further eliminates the ambiguity of tags.
# When using short names, there is always an inherent risk that the image being
# pulled could be spoofed. For example, a user wants to pull an image named
# `foobar` from a registry and expects it to come from myregistry.com. If
# myregistry.com is not first in the search list, an attacker could place a
# different `foobar` image at a registry earlier in the search list. The user
# would accidentally pull and run the attacker's image and code rather than the
# intended content. We recommend only adding registries which are completely
# trusted (i.e., registries which don't allow unknown or anonymous users to
# create accounts with arbitrary names). This will prevent an image from being
# spoofed, squatted or otherwise made insecure.  If it is necessary to use one
# of these registries, it should be added at the end of the list.
#
# # An array of host[:port] registries to try when pulling an unqualified image, in order.
unqualified-search-registries = ["docker.io"]
#
# [[registry]]
# # The "prefix" field is used to choose the relevant [[registry]] TOML table;
# # (only) the TOML table with the longest match for the input image name
# # (taking into account namespace/repo/tag/digest separators) is used.
# # 
# # The prefix can also be of the form: *.example.com for wildcard subdomain
# # matching.
# #
# # If the prefix field is missing, it defaults to be the same as the "location" field.
# prefix = "example.com/foo"
#
# # If true, unencrypted HTTP as well as TLS connections with untrusted
# # certificates are allowed.
# insecure = false
#
# # If true, pulling images with matching names is forbidden.
# blocked = false
#
# # The physical location of the "prefix"-rooted namespace.
# #
# # By default, this is equal to "prefix" (in which case "prefix" can be omitted
# # and the [[registry]] TOML table can only specify "location").
# #
# # Example: Given
# #   prefix = "example.com/foo"
# #   location = "internal-registry-for-example.net/bar"
# # requests for the image example.com/foo/myimage:latest will actually work with the
# # internal-registry-for-example.net/bar/myimage:latest image.
#
# # The location can be empty iff prefix is in a
# # wildcarded format: "*.example.com". In this case, the input reference will
# # be used as-is without any rewrite.
# location = internal-registry-for-example.com/bar"
#
# # (Possibly-partial) mirrors for the "prefix"-rooted namespace.
# #
# # The mirrors are attempted in the specified order; the first one that can be
# # contacted and contains the image will be used (and if none of the mirrors contains the image,
# # the primary location specified by the "registry.location" field, or using the unmodified
# # user-specified reference, is tried last).
# #
# # Each TOML table in the "mirror" array can contain the following fields, with the same semantics
# # as if specified in the [[registry]] TOML table directly:
# # - location
# # - insecure
# [[registry.mirror]]
# location = "example-mirror-0.local/mirror-for-foo"
# [[registry.mirror]]
# location = "example-mirror-1.local/mirrors/foo"
# insecure = true
# # Given the above, a pull of example.com/foo/image:latest will try:
# # 1. example-mirror-0.local/mirror-for-foo/image:latest
# # 2. example-mirror-1.local/mirrors/foo/image:latest
# # 3. internal-registry-for-example.net/bar/image:latest
# # in order, and use the first one that exists.

we need to figure out to the populate this file correctly for both kicbase and ISO in a unified way

[medyagh]

one might say we should not enable this to make users learn CRIO's default values,s since crio correctly advocates using the FQDN for less risk of spoofing... I am open to suggestions here

wdyt @tstromberg

[afbjorklund]

When did this get broken? It was supposed to have that configuration, since Kubernetes defaults to docker.io

https://github.com/kubernetes/minikube/blob/master/deploy/iso/minikube-iso/package/crio-bin/registries.conf

If you install CRI-O from the kubernetes repository, it will have both docker.io (and quay.io) added by default.

https://github.com/cri-o/packaging/blob/main/templates/latest/cri-o/bundle/registries.conf

[afbjorklund]

https://github.com/cri-o/cri-o/blob/main/install.md#validate-registries-in-registriesconf

The new syntax was introduced in cri-o/cri-o@2e49302 but it should mean the same thing

[medyagh]

so is that not the default behaviour of CRIO ? we install crio in kicbase

# install cri-o based on https://github.com/cri-o/cri-o/blob/release-1.24/README.md#installing-cri-o
RUN export ARCH=$(dpkg --print-architecture | sed 's/ppc64el/ppc64le/' | sed 's/armhf/arm-v7/') && \
    if [ "$ARCH" != "ppc64le" ] && [ "$ARCH" != "arm-v7" ]; then sh -c "echo 'deb https://downloadcontent.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/${CRIO_VERSION}/xUbuntu_22.04/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:${CRIO_VERSION}.list" && \
    curl -LO https://downloadcontent.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/${CRIO_VERSION}/xUbuntu_22.04/Release.key && \
    apt-key add - < Release.key && \
    clean-install cri-o cri-o-runc; fi

so is that not the default config ?

[afbjorklund]

Maybe it changed in the later versions, kubernetes 1.24 is quite old at this point - the latest cri-o release is 1.30

EDIT: Right, in the old packages the registries.conf was not in cri-o but in a separate containers-common deb.

It still had docker.io, though. (containers-common_100:1-22:all)

# # An array of host[:port] registries to try when pulling an unqualified image, in order.
unqualified-search-registries = ["docker.io", "quay.io"]

[medyagh]

I tried to install the latest version and as seen before in crio/podman, their installation instructions seems abandoned for ubuntu, its interesting through years been same story ...
cri-o/cri-o#8471

[afbjorklund]

You can see this blog post for the packaging changes, moving to Kubernetes:

https://kubernetes.io/blog/2023/10/10/cri-o-community-package-infrastructure/

[afbjorklund]

Current KIC image seem to have a non-working setup:

#
# # An array of host[:port] registries to try when pulling an unqualified image, in order.
# unqualified-search-registries = ["example.com"]
#

golang-github-containers-image: /etc/containers/registries.conf
golang-github-containers-common: /etc/containers/registries.conf.d/shortnames.conf

https://github.com/containers/image/blob/main/registries.conf

[afbjorklund]

This got broken here, with the upgrade from Ubuntu 20.04 to Ubuntu 24.04:

commit c2bb950 (minikube v1.31.0)

 RUN export ARCH=$(dpkg --print-architecture | sed 's/ppc64el/ppc64le/') && \
-    sh -c "echo 'deb https://downloadcontent.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" && \
-    curl -LO https://downloadcontent.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_20.04/Release.key && \
+    sh -c "echo 'deb https://downloadcontent.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_22.04/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" && \
+    curl -LO https://downloadcontent.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_22.04/Release.key && \
     apt-key add - < Release.key && \
     if [ "$ARCH" != "ppc64le" ]; then \
-        clean-install containers-common catatonit conmon containernetworking-plugins cri-tools podman-plugins crun; \
+        clean-install catatonit conmon cri-tools crun; \
     else \
-               clean-install containers-common conmon containernetworking-plugins crun; \
+        clean-install conmon crun; \
     fi

https://build.opensuse.org/projects/devel:kubic:libcontainers:stable/packages/containers-common/files/_service:obs_scm:registries.conf?expand=1

#
# # An array of host[:port] registries to try when pulling an unqualified image, in order.
unqualified-search-registries = ["registry.fedoraproject.org", "registry.access.redhat.com", "docker.io"]
#

[afbjorklund]

Here is the missing step, as hidden in the installlation documentation for cri-o:

https://github.com/cri-o/cri-o/blob/main/install.md#validate-registries-in-registriesconf

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten