Unable to pull image from local registry

[arvindkgs]

The 'demo' application fails to deploy on the kubernetes cluster when I run the command:
kubectl apply -f deployment.yml

Error response from daemon: Get http://127.0.0.1:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused

dockerd logs
INFO: server gave http response to https client

Google gives a possible fix

Add to /etc/docker/daemon.json => { "insecure-registries":["127.0.0.0/8","localhost:5000"] }

Restarted dockerd

docker info shows

Insecure Registries:

localhost:5000

127.0.0.0/8

Tried all steps above again, but got same error

Can you please tell me how to fix this?

Note: I am running on minikube on ArchLinux. I tried to run on kind, I don't get any error, but following command
curl localhost:8080

ERROR: Connection refused

How do I check logs for kind?

[arvindkgs]

With kind I tried to load image into cluster as:
kind load docker-image localhost:5000/apps/demo

However running kubectl cluster-info dump shows that the demo app is not started, with failure as:

 "containerStatuses": [
                    {
                        "name": "demo",
                        "state": {
                            "waiting": {
                                "reason": "ErrImagePull",
                                "message": "rpc error: code = Unknown desc = failed to pull and unpack image \"localhost:5000/apps/demo:latest\": failed to resolve reference \"localhost:5000/apps/demo:latest\": failed to do request: Head http://172.17.0.2:5000/v2/apps/demo/manifests/latest: dial tcp 172.17.0.2:5000: i/o timeout"
                            }
                        },
                        "lastState": {},
                        "ready": false,
                        "restartCount": 0,
                        "image": "localhost:5000/apps/demo",
                        "imageID": ""
                    }
                ]

My docker registry is running:
docker ps | grep registry

844c8b8a1e28 registry:2 "/entrypoint.sh /etc…" 18 hours ago Up 18 hours 0.0.0.0:5000->5000/tcp registry

NOTE: I started my docker registry as:
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2

[arvindkgs]

I got it to work on minikube. The issue is with minikube. You need to add flag for '--insecure-registry' when starting the minikube as:
minikube start --vm-driver=kvm2 --insecure-registry="ip:5000"

NOTE: localhost did not work. You need to specify the ip
ip addr and get ip address of your machine. More information here: kubernetes/minikube#604

Similarly you need to add above flag for docker, as mentioned above. More info: https://docs.docker.com/registry/insecure/

[js123331]

@arvindkgs how did you specify the insecure flag in the docker registry command?

Is it just: docker run -d -p 5000:5000 --restart=always --name registry registry:2?
docker run -d -p 5000:5000 --restart=always --name registry registry:2 --insecure-registry="ip_addr_of_machine:port number of application"?

And this insecure registry command must be run after adding the insecure registry to the /etc/docker/daemon.json file?
https://stackoverflow.com/questions/42211380/add-insecure-registry-to-docker

Thanks for the help.

[arvindkgs]

@js123331 you need to first start minikube, then edit /etc/docker/daemon.json and restart docker daemon. No need to specify the registry when starting the docker container

[js123331]

@arvindkgs

I tried the following procedure:

1. Edited the docker daemon json to add: "insecure-registries":["127.0.0.1","localhost:5000"] }
   My docker-daemon json looks like this:
   {"builder":{"gc":{"defaultKeepStorage":"20GB","enabled":true}},"experimental":false,"features":{"buildkit":true},"insecure-registries":["127.0.0.1","localhost:5000"] }

2. I re-started the docker daemon.

3. I Started the docker registry as: docker run -d -p 5000:5000 --restart=always --name registry registry:2.

4. I started minikube as: minikube start --kubernetes-version=v1.19.15 --insecure-registry="ip_address_of_machine:5000"

5. I then did some project specfic steps such as: building the image and updating the helm charts.

Build Step: eval $(minikube docker-env) &&
docker build .
--build-arg PROJECT_VERSION
-t localhost:5000/repository:latest

When I look at the statuses of my pods, I get:

When I describe that pod, I get: Failed to pull image "localhost:5000/repository:latest": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused.

Same error message as before.

Is there anything else I can change? Do I need to replace localhost/127.0.0.1 in the build step to ip_addr_of_my_machine.

thanks for the help!

[lbhuang2015]

@arvindkgs

I tried the following procedure:

1. Edited the docker daemon json to add: "insecure-registries":["127.0.0.1","localhost:5000"] }
   My docker-daemon json looks like this:
   {"builder":{"gc":{"defaultKeepStorage":"20GB","enabled":true}},"experimental":false,"features":{"buildkit":true},"insecure-registries":["127.0.0.1","localhost:5000"] }
2. I re-started the docker daemon.
3. I Started the docker registry as: docker run -d -p 5000:5000 --restart=always --name registry registry:2.
4. I started minikube as: minikube start --kubernetes-version=v1.19.15 --insecure-registry="ip_address_of_machine:5000"
5. I then did some project specfic steps such as: building the image and updating the helm charts.

Build Step: eval $(minikube docker-env) && docker build . --build-arg PROJECT_VERSION -t localhost:5000/repository:latest

When I look at the statuses of my pods, I get:

When I describe that pod, I get: Failed to pull image "localhost:5000/repository:latest": rpc error: code = Unknown desc = Error response from daemon: Get http://localhost:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused.

Same error message as before.

Is there anything else I can change? Do I need to replace localhost/127.0.0.1 in the build step to ip_addr_of_my_machine.

thanks for the help!

Hi, Did you solve it? I have the same problem, I'm running a minikube with the latest docker desktop for mac, and having a docker registry 2 on the same docker daemon. I could push or pull images to this private registry, but failed to pull image from pod, got the same error info: Failed to pull image "xxx:5000/repository:latest": rpc error: code = Unknown desc = Error response from daemon: Get http://xxx:5000/v2/: dial tcp 127.0.0.1:5000: connect: connection refused.