CODEWITHSUNDEEP
dockerkubernetesdocker-registry
bayman
bayman

Reputation: 1719

Kubernetes/docker ErrImagePull when deploying app from private registry

I am getting ErrImagePull when trying to create a deployment from an image hosted on my private helm docker registry. Based on the "server gave HTTP response to HTTPS client" error, I tried adding the --insecure-registry option in systemd drop-in as well as the daemon.json file on all worker nodes and master but it's still not working. What else should I try to troubleshoot?

edit: Perhaps it's because docker is using containerd?

    $ sudo systemctl status docker
    ● docker.service - Docker Application Container Engine
       Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
      Drop-In: /etc/systemd/system/docker.service.d
               └─docker.conf
       Active: active (running) since Fri 2020-02-07 10:00:25 UTC; 4min 44s ago
         Docs: https://docs.docker.com
     Main PID: 27700 (dockerd)
        Tasks: 14
       Memory: 41.8M
       CGroup: /system.slice/docker.service
               └─27700 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

    $ cat /etc/systemd/system/docker.service.d/docker.conf
    DOCKER_OPTS="--insecure-registry 10.10.30.200:5000"

    $ cat /etc/docker/daemon.json

    {
      "exec-opts": ["native.cgroupdriver=systemd"],
      "log-driver": "json-file",
      "log-opts": {
        "max-size": "100m"
      },
      "storage-driver": "overlay2",
      "insecure-registries": ["10.10.30.200:5000"]
    }    

    $ curl 10.10.30.200:5000/v2/mybuild/tags/list
        {"name":"mybuild","tags":["v1"]}

    $ kubectl describe pod myweb-769d57d99-lz6xs
    ...
    Normal   Pulling    1s (x2 over 13s)   kubelet, k8s-node2  Pulling image "10.10.30.200:5000/mybuild:v1"
    Warning  Failed     1s (x2 over 13s)   kubelet, k8s-node2  Failed to pull image "10.10.30.200:5000/mybuild:v1": rpc error: code = Unknown desc = failed to resolve image "10.10.30.200:5000/mybuild:v1": no available registry endpoint: failed to do request: Head https://10.10.30.200:5000/v2/mybuild/manifests/v1: http: server gave HTTP response to HTTPS client
    Warning  Failed     1s (x2 over 13s)   kubelet, k8s-node2  Error: ErrImagePull

    $ cat deployment.yaml

    ---
    apiVersion: apps/v1
    kind: Deployment                                                                                                                                         metadata:
      labels:
        app: myweb
      name: myweb
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: myweb
      template:
        metadata:
          labels:
            app: myweb
        spec:
          containers:
          - image: 10.10.30.200:5000/mybuild:v1
            imagePullPolicy: Always
            name: myweb

    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: myweb
      name: myweb
    spec:
      ports:
      - nodePort: 32223
        port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: myweb
      type: NodePort

Upvotes: 4

Views: 1807

Answers (1)

kool
kool

Reputation: 3613

I reproduced your issue and the solution is to add 

{
  "insecure-registry" : ["10.10.30.200:5000"]
}

on every node (workers and master) in /etc/docker/daemon.json in the cluster and restarting docker sudo systemctl restart docker to load new configuration.

You can follow this guide on how to set up insecure private docker registry using helm.

Once the chart was successfully installed, add entry in daemon.json on every node and restart docker. After that you can tag and push your image to the repository.

To check if the image was successfully pushed to your registry you can run:

curl -X GET http://10.10.30.200:5000/v2/_catalog

and the output should be similar to this:

{"repositories":["ubuntu/myimage"]}

Upvotes: 1

Related Questions