CODEWITHSUNDEEP
dockercontinuous-integrationgitlabdockerfilegitlab-ci
Lorenzo Musetta
Lorenzo Musetta

Reputation: 31

How to cache the base Docker image when using an external CI CD provider

I am currently using a CI CD provider (gitlab.com) to build docker images (via multi-stage) and then deploy those built images to my server.

The images are very similar between each other, in fact they all use php-apache as a base image; the only thing that changes is the code that I COPY from the repository.

So in theory, the best scenario for disk usage would be for my images to all share the same cached version of php-apache.

However, on my server I can see that the resulting image is always 400MB+, which means that probably the base php-apache is not cached at all.

What can I do to cache the base image php-apache for all of my built images? Ideally, every app container should be of lesser size because they all share the same image. Is this even possible? Or maybe it is possible but only by building locally instead than on ci cd? Thanks.

Just for clarity, this is a simplified version of my Dockerfile:

FROM jitesoft/composer:7.4 AS php_installer
COPY ./ /app
WORDIR app
RUN composer install

FROM php:7.4-apache
COPY --from=php_installer /app/. /var/www/

Instead, the gitlab deploy happens via a ci cd script very similar to this https://caprover.com/docs/ci-cd-integration.html#9--gitlab-ci-file

Upvotes: 1

Views: 352

Answers (1)

Jack
Jack

Reputation: 521

I am assuming you are seeing the size of docker images through docker image list, this can be misleading as this is the total size needed to install all the layers of the image, what does that mean?

Layers

Docker images are not monoliths, instead they are composed of read-only layers, which are ways of representing changes on the filesystem. You can see the layers that compose your image by doing:

docker history <image id>

Example:

Dockerfile:

FROM ubuntu:16.04

RUN echo "Hi" >> test.txt

> docker build -t test:latest .
> docker image list 
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
test                    latest              06c4055eba9f        3 minutes ago       132MB
ubuntu                  16.04               f6f49faac5cf        6 days ago          132MB
> docker history 06c4055eba9f
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
06c4055eba9f        3 minutes ago       /bin/sh -c echo "Hi" >> test.txt                3B             
f6f49faac5cf        6 days ago          /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B             
<missing>           6 days ago          /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B             
<missing>           6 days ago          /bin/sh -c rm -rf /var/lib/apt/lists/*          0B             
<missing>           6 days ago          /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B           
<missing>           6 days ago          /bin/sh -c #(nop) ADD file:925571658dd8453e5…   132MB

Here we can see that this image that contains a file that says Hi (which is three bytes long "Hi\0") consists of two layers. One that is just over 132Mb and one that is 3B.

Why this matters

This is the magic of docker! When someone downloads your docker image they only need to download the layers that they don't have! This means that if someone was to download my test:latest image then they would only need to download 3B (assuming they have ubuntu:16.04).

The reason docker presents this 400Mb size to you is because that is the maximum size of the download if the user needs to download all the layers of the docker image.

If you want to read more about this here is a documentation page on it:

https://docs.docker.com/storage/storagedriver/

And the key part is:

Notice that all the layers are identical except the top layer of the second image. All the other layers are shared between the two images, and are only stored once in /var/lib/docker/. The new layer actually doesn’t take any room at all, because it is not changing any files, but only running a command.

Also you should read this article:

https://semaphoreci.com/blog/2018/03/14/docker-image-size.html

Conclusion

You don’t have to cache your base image because docker automatically does this for you. It was difficult to see that it was doing it for you though.

Upvotes: 1

Related Questions