CODEWITHSUNDEEP
dockercontainersdockerfilegoogle-cloud-build
Matthew Goslett
Matthew Goslett

Reputation: 1217

Is it possible to cache multi-stage docker builds?

I recently switched to multi-stage docker builds, and it doesn't appear that there's any caching on intermediate builds. I'm not sure if this is a docker limitation, something which just isn't available or whether I'm doing something wrong.

I am pulling down the final build and doing a --cache-from at the start of the new build, but it always runs the full build.

Upvotes: 60

Views: 33464

Answers (3)

Eric Greveson
Eric Greveson

Reputation: 281

Since the previous answer was posted, there is now a solution using the BuildKit backend:

This involves passing the argument --build-arg BUILDKIT_INLINE_CACHE=1 to your docker build command. You will also need to ensure BuildKit is being used by setting the environment variable DOCKER_BUILDKIT=1 (on Linux; I think BuildKit might be the default backend on Windows when using recent versions of Docker Desktop). A complete command line solution for CI might look something like:

export DOCKER_BUILDKIT=1
    
# Use cache from remote repository, tag as latest, keep cache metadata
docker build -t yourname/yourapp:latest \
      --cache-from yourname/yourapp:latest \
      --build-arg BUILDKIT_INLINE_CACHE=1 .
    
# Push new build up to remote repository replacing latest
docker push yourname/yourapp:latest

Some of the other commenters are asking about docker-compose. It works for this too, although you need to additionally specify the environment variable COMPOSE_DOCKER_CLI_BUILD=1 to ensure docker-compose uses the docker CLI (with BuildKit thanks to DOCKER_BUILDKIT=1) and then you can set BUILDKIT_INLINE_CACHE: 1 in the args: section of the build: section of your YAML file to ensure the required --build-arg is set.

For reference:

Upvotes: 28

Felixoid
Felixoid

Reputation: 593

I'd like to add another important point to the answer

Wrong answer

--build-arg BUILDKIT_INLINE_CACHE=1 caches only the last layer, and works only in cases when nothing is changed in the whole Dockerfile

So, to enable the caching of layers for the whole build, this argument should be replaced by --cache-to type=inline,mode=max. See the documentation

Correct answer

The documentation above as on 2023-03-28 states:

When generating a cache output, the --cache-to argument accepts a mode option for defining which layers to include in the exported cache. This is supported by all cache backends except for the inline cache.

That means, for the cached intermediate states with all layers one needs to use registry cache backend.

I plan to use the same image:tag as pushed at the build time, but with suffix -buildcache. So, --cache-from type=registry,ref=org/image:tag-buildcache,mode=max --cache-to type=registry,ref=org/image:tag-buildcache,mode=max

Upvotes: 10

Matthew Goslett
Matthew Goslett

Reputation: 1217

This appears to be a limitation of docker itself and is described under this issue - https://github.com/moby/moby/issues/34715

The workaround is to:

  1. Build the intermediate stages with a --target
  2. Push the intermediate images to the registry
  3. Build the final image with a --target and use multiple --cache-from paths, listing all the intermediate images and the final image
  4. Push the final image to the registry
  5. For subsequent builds, pull the intermediate + final images down from the registry first

Upvotes: 60

Related Questions