Getting 'kubelet --version': exit status 255 even after following all the fixes

[red8888]

What Happened?

Related #12274

% minikube version
minikube version: v1.29.0
commit: ddac20b4b34a9c8c857fc602203b6ba2679794d3

% docker version
Client:
 Cloud integration: v1.0.31
 Version:           20.10.23
 API version:       1.41
 Go version:        go1.18.10
 Git commit:        7155243
 Built:             Thu Jan 19 17:35:19 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux
 Experimental:      true

Server: Docker Desktop 4.17.0 (99724)
 Engine:
  Version:          20.10.23
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.10
  Git commit:       6051f14
  Built:            Thu Jan 19 17:31:28 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

I initially installed via homebrew and got this error.

I uninstalled via brew and reinstall the right arch:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-arm64
sudo install minikube-darwin-arm64 /usr/local/bin/minikube

I ran minikube delete --all --purge
I deleted the ~/.minikube directory

But starting a cluster I got this error:
minikube start -p mycluster --kubernetes-version=v1.23.14 --driver=docker --force-systemd=true
[ERROR KubeletVersion]: couldn't get kubelet version: cannot execute 'kubelet --version': fork/exec /var/lib/minikube/binaries/v1.23.14/kubelet: no such file or directory

I'm at a loss, what else can I try? This is suppose to work after purging but its not for me.

Attach the log file

% minikube logs -p mycluster --file=logs.txt
E0317 11:23:33.593922 85617 logs.go:193] command /bin/bash -c "sudo /var/lib/minikube/binaries/v1.23.14/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig" failed with error: /bin/bash -c "sudo /var/lib/minikube/binaries/v1.23.14/kubectl describe nodes --kubeconfig=/var/lib/minikube/kubeconfig": Process exited with status 1
stdout:

stderr:
The connection to the server localhost:8443 was refused - did you specify the right host or port?
output: "\n** stderr ** \nThe connection to the server localhost:8443 was refused - did you specify the right host or port?\n\n** /stderr **"

❗ unable to fetch logs for: describe nodes

There is no /var/lib/minikube folder on my system

Operating System

macOS (Default)

Driver

Docker

[red8888]

Ugh I didn't even consider this: #14276

I also had DOCKER_DEFAULT_PLATFORM=linux/amd64 set.

Maybe minikube should be smart enough unset this or pass docker the platform arg specifically?

This is a common setting for users and especially M1 users I'd think because we need to build and run images with the same arch we deploy to production (to test locally consistently).

I would have expected DOCKER_DEFAULT_PLATFORM to only effect my own docker builds. Is minikubet orchestrating the docker cli itself? Curious how this setting is effecting minikube- maybe it effects all docker pulls run from the docker cli or api?