How to use local docker images in kubernetes deployments (NOT minikube)

Question

I have a VM with kubernetes installed using kubeadm (NOT minikube). The VM acts a the single node of the cluster, with taints removed to allow it to act as both Master and Worker node (as shown in the kubernetes documentation). I have saved, transfered and loaded a my app:test image into it. I can easily run a container with it using docker run. It shows up when I run sudo docker images.

When I create a deployment/pod that uses this image and specify Image-PullPolicy: IfNotPresent or Never, I still have the ImagePullBackoff error. The describe command shows me it tries to pull the image from dockerhub...

Note that when I try to use a local image that was pulled as the result of creating another pod, the ImagePullPolicies seem to work, no problem. Although the image doesn't appear when i run sudo docker images --all.

How can I use a local image for pods in kubernetes? Is there a way to do it without using a private repository?

Answer 1

In my case, I have setup k8s cluster with kubeadm. so, if you want to use local image to run pod do below things.

If you have docker image, create tar file of image by using below command.

docker save <image-name> -o <filename.tar>

If you have containerd image, create tar file of image by using below command.

ctr image export <output-filename> <image-name>

After transferring the standalone archives to the other systems (using whatever means you prefer; I used scp), then load (or import) the images into containerd with this command:

ctr -n=k8s.io images import <filename-from-previous-step>

Now you can see image.

sudo crictl images

Answer 2

Here is another solution that worked for me.

The easiest solutions was in the end to install podman and load the image. I'm running on manjaro, using CRI-O as my CRI and kubeadm as control-plane.

sudo pacman -Sy podman 

save your image from docker to a .tar file and load the image using sudo podman. (Note: running podman with sudo is necessary)

sudo podman load -i your-local-image.tar

It instantly showed up in the CRI-image registery when running

sudo crictl images

Example to define your kubernetes job:

#local-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: local-job
spec:
  template:
    spec:
      containers:
      - name: local-img-pod
        image: localhost/<Your-image-tag> 
        command: ["./your-app-binary","parameter1"]
        imagePullPolicy: Never
      restartPolicy: Never
  backoffLimit: 4

and start the job with

kubectl apply -f local-job.yaml

Answer 3

One can do so with a combination of crictl and ctr, if using containerd.

TLDR: these steps, which are also described in the crictl github documentation:

1- Once you get the image on the node (in my case, a VM), make sure it is in an archive (.tar). You can do that with the docker save or ctr image export commands.

2- Use sudo ctr -n=k8s.io images import myimage.tar while in the same directory as thearchived image to add it to containerd in the namespace that kubernetes uses to track it's images. It should now appear when you run sudo crictl images.

---

As suggested, I tried listing images with crictl and my app:test did not appear. However, trying to import my local image through crictl didn't seem to work either. I used crictl pull app:test and it showed the following error message:

FATA[0000] pulling image failed: rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/app:test": failed to resolve reference "docker.io/library/app:test": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed.

However, when following these steps, my image is finally recognized as an existing local image in kubernetes. They are actually the same as suggested in the crictl github documentation

How does one explain this? How do images get "registered" in the kubernetes cluster? Why couldn't crictl import the image? I might post another issue to ask that...

Answer 4

image doesn't appear when i run sudo docker images --all

Based on your comment, you are using K8s v1.22, which means it is likely your cluster is using containerd container runtime instead of docker (you can check with kubectl get nodes -o wide, and see the last column).

Try listing your images with crictl images and pulling with crictl pull <image_name> to preload the images on the node.

Answer 5

Your cluster is bottled inside of your VM, so what you call local will always be remote for that cluster in that VM. And the reason that kubernetes is trying to pull those images, is because it can't find them in the VM.

Dockerhub is the default place to download containers from, but you can set kubernetes to pull from aws (ECR) from azure (ACR), from github packages (GCR) and from your own private server.

You've got about 100 ways to solve this, none of them are easy or will just work.

1 - easiest, push your images to Dockerhub and let your cluster pull from it.

2 - setup a local private container registry and set your kubernetes VM to pull from it (see this)

3 - setup a private container registry in your kubernetes cluster and setup scripts in your local env to push to it (see this)