how can we pull images from a private registry from colima kubernetes?

[apple-corps]

My current .docker/config.json looks like below, but alternatively I could configure a credentials helper. However I am unable to pull images from the colima configured kubernetes from the private registry. How can we pull images from a private registry from colima kubernetes?

{
        "auths": {
                "my-reg-1": {
                        "auth": "some-value"
                },
                "my-reg-2": {
                        "auth": "a-different-value"
                }
        },
        "currentContext": "colima"
}

[mgs255]

Hi - I'm trying to achieve the same sort of thing and have spent quite some time fiddling trying to get this to work. I've made some progress, but there are some issues I'd like to overcome.

First off I'm using an MacOS 13.5, trying to use:

• colima and lima installed via brew at HEAD
• kubernetes 1.24.x
• containerd
• a private registry which in our case is a Harbor registry proxy (https://goharbor.io/) pulling through to ECR.
• traefik ingress disabled (was eventually intending on using nginx)

According to the K3S docs, the private registry configuration should be automatically picked up from /etc/rancher/k3s/registries.yaml. However this doesn't seem to work for me. However, if I stop the k3s service: sudo rc-service k3s stop, then manually run the k3s service using:

sudo /usr/local/bin/k3s server --debug \
    --private-registry /etc/rancher/k3s/registries.yaml \
    --disable traefik

That does allow me to pull down images from ECR using the registry config:

mirrors:
  <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com:
    endpoint:
     - "https://harbor.mycompany.com"
    rewrite:
      "^(.*)": "myproject/$1"
configs:
  "harbor.mycompany.com":
    auth:
      username: <username>
      password: <harbor_cli_token>

I'd be interested to know if anyone is able to progress this further. It would be really useful to be able to get this working for local development. For completeness I'm using the following colima configuration

# Number of CPUs to be allocated to the virtual machine.
# Default: 2
cpu: 2

# Size of the disk in GiB to be allocated to the virtual machine.
# NOTE: changing this has no effect after the virtual machine has been created.
# Default: 60
disk: 60

# Size of the memory in GiB to be allocated to the virtual machine.
# Default: 2
memory: 2

# Architecture of the virtual machine (x86_64, aarch64, host).
# Default: host
arch: aarch64

# Container runtime to be used (docker, containerd).
# Default: docker
runtime: containerd

# Kubernetes configuration for the virtual machine.
kubernetes:
  # Enable kubernetes.
  # Default: false
  enabled: true
  
  # Kubernetes version to use.
  # This needs to exactly match a k3s version https://github.com/k3s-io/k3s/releases
  # Default: latest stable release
  version: v1.24.15+k3s1
  
  # Additional args to pass to k3s https://docs.k3s.io/cli/server
  # Default: traefik is disabled
  k3sArgs:
    - --disable traefik
    - --private-registry /etc/rancher/k3s/registries.yaml

# Auto-activate on the Host for client access.
# Setting to true does the following on startup
#  - sets as active Docker context (for Docker runtime).
#  - sets as active Kubernetes context (if Kubernetes is enabled).
# Default: true
autoActivate: true

# Network configurations for the virtual machine.
network:
  # Assign reachable IP address to the virtual machine.
  # NOTE: this is currently macOS only and ignored on Linux.
  # Default: false
  address: true
  
  # Custom DNS resolvers for the virtual machine.
  #
  # EXAMPLE
  # dns: [8.8.8.8, 1.1.1.1]
  #
  # Default: []
  dns: []
  
  # DNS hostnames to resolve to custom targets using the internal resolver.
  # This setting has no effect if a custom DNS resolver list is supplied above.
  # It does not configure the /etc/hosts files of any machine or container.
  # The value can be an IP address or another host.
  #
  # EXAMPLE
  # dnsHosts:
  #   example.com: 1.2.3.4
  dnsHosts:
    host.docker.internal: host.lima.internal
  
  # Network driver to use (slirp, gvproxy), (requires vmType `qemu`)
  #   - slirp is the default user mode networking provided by Qemu
  #   - gvproxy is an alternative to VPNKit based on gVisor https://github.com/containers/gvisor-tap-vsock
  # Default: gvproxy
  driver: gvproxy

# ===================================================================== #
# ADVANCED CONFIGURATION
# ===================================================================== #

# Forward the host's SSH agent to the virtual machine.
# Default: false
forwardAgent: false

# Docker daemon configuration that maps directly to daemon.json.
# https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file.
# NOTE: some settings may affect Colima's ability to start docker. e.g. `hosts`.
#
# EXAMPLE - disable buildkit
# docker:
#   features:
#     buildkit: false
#
# EXAMPLE - add insecure registries
# docker:
#   insecure-registries:
#     - myregistry.com:5000
#     - host.docker.internal:5000
#
# Colima default behaviour: buildkit enabled
# Default: {}
docker: {}

# Virtual Machine type (qemu, vz)
# NOTE: this is macOS 13 only. For Linux and macOS <13.0, qemu is always used.
#
# vz is macOS virtualization framework and requires macOS 13
#
# Default: qemu
vmType: vz

# Utilise rosetta for amd64 emulation (requires m1 mac and vmType `vz`)
# Default: false
rosetta: false

# Volume mount driver for the virtual machine (virtiofs, 9p, sshfs).
#
# virtiofs is limited to macOS and vmType `vz`. It is the fastest of the options.
#
# 9p is the recommended and the most stable option for vmType `qemu`.
#
# sshfs is faster than 9p but the least reliable of the options (when there are lots
# of concurrent reads or writes).
#
# Default: virtiofs (for vz), sshfs (for qemu)
mountType: virtiofs

# Propagate inotify file events to the VM.
# NOTE: this is experimental.
mountInotify: false

# The CPU type for the virtual machine (requires vmType `qemu`).
# Options available for host emulation can be checked with: `qemu-system-$(arch) -cpu help`.
# Instructions are also supported by appending to the cpu type e.g. "qemu64,+ssse3".
# Default: host
cpuType: host

# For a more general purpose virtual machine, Ubuntu container is optionally provided
# as a layer on the virtual machine.
# The underlying virtual machine is still accessible via `colima ssh --layer=false` or running `colima` in
# the Ubuntu session.
#
# Default: false
layer: false

# Custom provision scripts for the virtual machine.
# Provisioning scripts are executed on startup and therefore needs to be idempotent.
#
# EXAMPLE - script exected as root
# provision:
#   - mode: system
#     script: apk add htop vim
#
# EXAMPLE - script exected as user
# provision:
#   - mode: user
#     script: |
#       [ -f ~/.provision ] && exit 0;
#       echo provisioning as $USER...
#       touch ~/.provision
#
# Default: []
provision: []

# Modify ~/.ssh/config automatically to include a SSH config for the virtual machine.
# SSH config will still be generated in ~/.colima/ssh_config regardless.
# Default: true
sshConfig: true

# Configure volume mounts for the virtual machine.
# Colima mounts user's home directory by default to provide a familiar
# user experience.
#
# EXAMPLE
# mounts:
#   - location: ~/secrets
#     writable: false
#   - location: ~/projects
#     writable: true
#
# Colima default behaviour: $HOME and /tmp/colima are mounted as writable.
# Default: []
mounts: []

# Environment variables for the virtual machine.
#
# EXAMPLE
# env:
#   KEY: value
#   ANOTHER_KEY: another value
#
# Default: {}
env: {}