[abctl] local install failing with No such container: airbyte-abctl-control-plane

[PabloPardoGarcia]

What happened?

I get the following error when trying to install airbyte locally with abctl:

> abctl local install                                  
  INFO    Using Kubernetes provider:
            Provider: kind
            Kubeconfig: /Users/pablopardogarcia/.airbyte/abctl/abctl.kubeconfig
            Context: kind-airbyte-abctl
 SUCCESS  Found Docker installation: version 27.1.1                                                                                                                                                                                
 SUCCESS  Port 8000 appears to be available                                                                                                                                                                                        
  INFO    No existing cluster found, cluster 'airbyte-abctl' will be created                                                                                                                                                       
  ERROR   Cluster 'airbyte-abctl' could not be created                                                                                                                                                                             
  ERROR   unable to create kind cluster: failed to init node with kubeadm: command "docker exec --privileged airbyte-abctl-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1

When running the failing command, I get this other error:

> docker exec --privileged airbyte-abctl-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6
Error response from daemon: No such container: airbyte-abctl-control-plane

This issue was pointed out in this comment as well: #41938 (comment)

What did you expect to happen?

Install works as expected

Abctl Version

$ abctl version
version: v0.13.1

Docker Version

$ docker version
Client:
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Tue Jul 23 19:54:12 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:14 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

OS Version

# On Mac:
$ uname -a
Darwin Pablos-MBP.fritz.box 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6030 arm64

[tom-mont]

I am receiving the same issue and receive the same response when running docker exec --privileged airbyte-abctl-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6

[marcosmarxm]

@PabloPardoGarcia and @tom-mont are you running docker as root?

[PabloPardoGarcia]

@marcosmarxm no, this is the output of docker info command:

docker info
Client:
 Version:    27.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.1-desktop.1
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.1-desktop.1
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.14
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.11.0
    Path:     /Users/pablopardogarcia/.docker/cli-plugins/docker-scout

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 44
 Server Version: 27.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.0-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 11
 Total Memory: 7.655GiB
 Name: docker-desktop
 ID: ffec7027-0f75-4ab9-a074-21ce83fe402b
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/pablopardogarcia/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

[PabloPardoGarcia]

@marcosmarxm I managed to make it work!

The problem was that docker was using the wrong platform to build the images, but exporting the env variable DOCKER_DEFAULT_PLATFORM=linux/arm64 it worked like a charm 😊

[tom-mont]

Hey @PabloPardoGarcia do you mind explaining what you mean by exporting the variable?

[PabloPardoGarcia]

Run this:

export DOCKER_DEFAULT_PLATFORM=linux/arm64
abctl local install

This way you are telling docker to build the airbyte images for arm64 architecture.

[tom-mont]

Heads up @PabloPardoGarcia , I was able to fix the issue by temporarily upgrading the size of my EC2 instance

[tom-mont]

I was also able to resolve the issue by rebooting the EC2 instance