CODEWITHSUNDEEP
docker-composegitlabgitlab-ci-runner
yahop19531
yahop19531

Reputation: 223

gitlab runner - how to cache images rather than build every time?

I have setup gitlab runner to make use of the docker/compose:latest image to run a set of services.

By setting pull_policy="if-not-present" in the config.toml the docker/compose:latest image is cached and not pulled each time.

How can I set the docker service used in the test stage to use cached images (i.e. those being started using docker-compose up -d )? I'm a bit confused by the dind service.

.gitlab-ci.yml

variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_TLS_CERTDIR: ""
  DOCKER_DRIVER: overlay2
  CI_REGISTRY: my-docker-registry.local:5000

image:
  name: docker/compose:latest

services:
  - name: docker:dind
    command: ["--insecure-registry=my-docker-registry.local:5000"]

test:
  stage: test
  script:
    - docker-compose up -d

/etc/gitlab-runner/config.toml as follows:

                                                   concurrent = 1
check_interval = 0

[[runners]]
  name = "user1"
  url = "http://gitlab.example.local/"
  token = "<MYTOKEN>"
  executor = "docker"
  [runners.docker]
    extra_hosts = ["my-docker-registry.local:192.168.1.100"]
    tls_verify = false
    tls_verify = false
    image = "docker:latest"
    privileged = true
    disable_cache = false
    pull_policy="if-not-present"
    volumes = ["/certs/client", "/cache"]    
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    shm_size = 0
  [runners.cache]

Upvotes: 4

Views: 4290

Answers (1)

sytech
sytech

Reputation: 41119

The pull_policy only applies to images that the runner itself pulls. I.E. images you set in image: keys (or services[]:name).

If you use docker inside a job (e.g. where you use docker-compose up -d) when using the docker executor, then you cannot take advantage of cached layers and must pull the images from the registry first or build from scratch.

The reason for this is that docker-in-docker GitLab jobs use the docker:dind service as its docker daemon. This service comes up with a fresh state every time and therefore will not take advantage of cached images, even if the runner itself has the image cached.

You can, however, pull an image from your registry and use its as caches for your build using the --cache-from argument to docker build.

script:
    - docker pull $CI_REGISTRY_IMAGE:latest || true
    - docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .

You still have to pull the image each time, of course, but this may be faster than building, especially if your image installs many dependencies or compiles large modules or something like that.

See the docker-in-docker section in the docs for more info.

Upvotes: 4

Related Questions