Can I run an intermediate layer of a Docker image?

Question

When I fetch a Docker image from a repository I see that it pulls a bunch of layers with some ids, but when I try to run them using docker run it tells me that it can't find it.

Can I run a certain layer of a Docker image that I can see through docker history < image_id >, like a paleontologist digging to find something interesting?

What I tried:

docker pull ruby
Using default tag: latest
latest: Pulling from library/ruby
693502eb7dfb: Already exists
081cd4bfd521: Already exists
5d2dc01312f3: Already exists
54a5f7da9a4f: Pulling fs layer
168cf3f33330: Pulling fs layer
021d84fef638: Pulling fs layer
168c3c107cd1: Waiting
f001b782a027: Waiting

Then:

docker run --rm -it 5d2dc01312f3 bash
Unable to find image '5d2dc01312f3:latest' locally

The question popped to me when I was pulling an image that I have build and published. I could see it clearly that one layer had almost 1.2 GB of size. It was the layer where I copied some files from the machine I used to build the image and on the next layer I cleaned and deleted some of those files as I was cleaning the image, but the total size of the image was about 1.5 GB which means that the big fat layer is of 1.2 GB is there. I just want to see if I can peek into that fat layer.

Answer 1

Comment out the Dockerfile where I want to test it.

docker build -t broken .

Original:

FROM node:${NODE_VERSION}-slim as base

WORKDIR /app
COPY . .

# this line is broken
RUN npm install

And now for the "intermediate container":

docker build -t intermediate .

"Intermediate" container:

FROM node:${NODE_VERSION}-slim as base

WORKDIR /app
COPY . .

# Commented out this broken line
# so i can debug what was copied in
#RUN npm install

Now I get an image, and I can exec/run -it.

Older versions had the intermediate container images available to run. Like originally asked, The newer versions do not.

Answer 2

Well, you can run an associated image if the final image was built locally on the same host machine. Each layer would have an intermediate image associated with it. You can show these layers and their associated images by running docker history <image-id>

Images that are pulled from a registry won't have that. In that case, docker history will show the value <missing> to indicate that there are no images associated with these layers. Hence there is nothing to run.

The digests on the output of docker pull are digests of the layers not intermediate images and can't be run.

Here is a typical scenario:

1. You build an image on a host: An intermediate image will be created for each layer (this is done for docker build cache)
2. You push the image to a registry: Only the final image is pushed (which references all the layers), but none of the intermediate images are pushed.
3. You pull the image on another host: Only the final image is available now on that other Docker host.

You can read more about layers at explaining-docker-image-ids.

Answer 3

The most popular tool for examining image layers is currently dive. It automates the docker save and a filesystem explorer in the CLI. By default, it's examining images from the local docker engine, so it typically needs access to the docker socket, e.g.:

docker run --rm -it \
    -v /var/run/docker.sock:/var/run/docker.sock \
    wagoodman/dive:latest ${your_image_name}

Answer 4

So many years ahead, I came back to answer this question. To peek at the intermediate layers the best and easiest solution is to export the docker image into archived format and than every layer will be a tar on its own inside the bigger tar which is of the docker image.

Let's take for example docker image of tomcat:9.0 To export it into tar format:

docker save tomcat:9.0 >tomcat-9.0.tar

When exported, inside it there can be found many Layers , each one is archived in its own tar file:

tar -tvf tomcat-9.0.tar 
drwxr-xr-x 0/0               0 2022-09-02 12:41 1fdef6bff31128d5a1a41bdd5fea06be032ef59fd985da09842a252310b4afa3/
-rw-r--r-- 0/0               3 2022-09-02 12:41 1fdef6bff31128d5a1a41bdd5fea06be032ef59fd985da09842a252310b4afa3/VERSION
-rw-r--r-- 0/0             482 2022-09-02 12:41 1fdef6bff31128d5a1a41bdd5fea06be032ef59fd985da09842a252310b4afa3/json
-rw-r--r-- 0/0        20891136 2022-09-02 12:41 1fdef6bff31128d5a1a41bdd5fea06be032ef59fd985da09842a252310b4afa3/layer.tar
drwxr-xr-x 0/0               0 2022-09-02 12:41 2ecda9f325b8b38fc5e0a2f1fed4526c517c087e909875abdf1ee4fbcab37b30/
-rw-r--r-- 0/0               3 2022-09-02 12:41 2ecda9f325b8b38fc5e0a2f1fed4526c517c087e909875abdf1ee4fbcab37b30/VERSION
-rw-r--r-- 0/0             482 2022-09-02 12:41 2ecda9f325b8b38fc5e0a2f1fed4526c517c087e909875abdf1ee4fbcab37b30/json
-rw-r--r-- 0/0       329143808 2022-09-02 12:41 2ecda9f325b8b38fc5e0a2f1fed4526c517c087e909875abdf1ee4fbcab37b30/layer.tar
drwxr-xr-x 0/0               0 2022-09-02 12:41 2fe9b9ee40730bab9cf86153dd37fbddb4990dd09ba5b9f22e55eeae13063fe8/
-rw-r--r-- 0/0               3 2022-09-02 12:41 2fe9b9ee40730bab9cf86153dd37fbddb4990dd09ba5b9f22e55eeae13063fe8/VERSION
-rw-r--r-- 0/0             406 2022-09-02 12:41 2fe9b9ee40730bab9cf86153dd37fbddb4990dd09ba5b9f22e55eeae13063fe8/json
-rw-r--r-- 0/0        80354304 2022-09-02 12:41 2fe9b9ee40730bab9cf86153dd37fbddb4990dd09ba5b9f22e55eeae13063fe8/layer.tar
drwxr-xr-x 0/0               0 2022-09-02 12:41 3fff0862317ff87f1d757557325c158d405afd7fd98f31544ce10cf3ad2bdaa7/
-rw-r--r-- 0/0               3 2022-09-02 12:41 3fff0862317ff87f1d757557325c158d405afd7fd98f31544ce10cf3ad2bdaa7/VERSION
-rw-r--r-- 0/0             482 2022-09-02 12:41 3fff0862317ff87f1d757557325c158d405afd7fd98f31544ce10cf3ad2bdaa7/json
-rw-r--r-- 0/0            3072 2022-09-02 12:41 3fff0862317ff87f1d757557325c158d405afd7fd98f31544ce10cf3ad2bdaa7/layer.tar
drwxr-xr-x 0/0               0 2022-09-02 12:41 5cab171b0d1115044b08690329f32a7d99a546777097e935733a837100ef34c5/
-rw-r--r-- 0/0               3 2022-09-02 12:41 5cab171b0d1115044b08690329f32a7d99a546777097e935733a837100ef34c5/VERSION
-rw-r--r-- 0/0             482 2022-09-02 12:41 5cab171b0d1115044b08690329f32a7d99a546777097e935733a837100ef34c5/json
-rw-r--r-- 0/0            3072 2022-09-02 12:41 5cab171b0d1115044b08690329f32a7d99a546777097e935733a837100ef34c5/layer.tar
drwxr-xr-x 0/0               0 2022-09-02 12:41 6b87b21c947802c9f53a614692b4369cc2d7aa951f64bf28cd1f8713e191894e/
-rw-r--r-- 0/0               3 2022-09-02 12:41 6b87b21c947802c9f53a614692b4369cc2d7aa951f64bf28cd1f8713e191894e/VERSION
-rw-r--r-- 0/0             482 2022-09-02 12:41 6b87b21c947802c9f53a614692b4369cc2d7aa951f64bf28cd1f8713e191894e/json
-rw-r--r-- 0/0        50671616 2022-09-02 12:41 6b87b21c947802c9f53a614692b4369cc2d7aa951f64bf28cd1f8713e191894e/layer.tar
drwxr-xr-x 0/0               0 2022-09-02 12:41 895adbb30677e7148af6d01e295347bd6ddc7cd5d929ee8fc3d03aedac5fa857/
-rw-r--r-- 0/0               3 2022-09-02 12:41 895adbb30677e7148af6d01e295347bd6ddc7cd5d929ee8fc3d03aedac5fa857/VERSION
-rw-r--r-- 0/0            2588 2022-09-02 12:41 895adbb30677e7148af6d01e295347bd6ddc7cd5d929ee8fc3d03aedac5fa857/json
-rw-r--r-- 0/0            2048 2022-09-02 12:41 895adbb30677e7148af6d01e295347bd6ddc7cd5d929ee8fc3d03aedac5fa857/layer.tar
-rw-r--r-- 0/0           12431 2022-09-02 12:41 d4488b7f8c9b73c0f2e91f5e37990905ae3d34d55ea893b5e70b76fc7a2db1b7.json
-rw-r--r-- 0/0             661 1970-01-01 01:00 manifest.json
-rw-r--r-- 0/0              86 1970-01-01 01:00 repositories

To see and peek at the content of each layer, all what is left now is to decompress the tomcat tar and than decompress the tar of the targeted layer.

tar -xvf tomcat-9.0.tar
tar -tvf 6b87b21c947802c9f53a614692b4369cc2d7aa951f64bf28cd1f8713e191894e/layer.tar 
drwxr-xr-x 0/0               0 2022-09-02 07:24 etc/
drwxr-xr-x 0/0               0 2022-09-02 07:24 etc/ca-certificates/
-rwxr-xr-x 0/0               0 1970-01-01 01:00 etc/ca-certificates/.wh..wh..opq
drwxr-xr-x 0/0               0 2021-10-16 18:09 etc/ca-certificates/update.d/
-rw-r--r-- 0/0            5529 2022-09-02 07:24 etc/ca-certificates.conf
drwxr-xr-x 0/0               0 2022-09-02 07:24 etc/default/
-rw-r--r-- 0/0              35 2022-09-02 07:24 etc/default/locale
drwxr-xr-x 0/0               0 2022-09-02 07:24 etc/fonts/
-rwxr-xr-x 0/0               0 1970-01-01 01:00 etc/fonts/.wh..wh..opq
drwxr-xr-x 0/0               0 2022-09-02 07:24 etc/fonts/conf.avail/
[...]

Answer 5

Yes, this is possible, but if you are using Docker 1.10 or later, only with self build images. E.g., docker history ruby will output <missing> for all but the topmost layer, because they doesn't have a tag.

If you build it yourself they will have a tag and you can start them like normal.

Answer 6

You can run an intermediate image of a Docker layer, which is probably what you want.

During your build you might need to inspect an image at certain point (step) in the build process e.g. in your Docker build output you'll see:

Step 17/30 : RUN rm -rf /some/directory
---> Running in 5ab963f2b48d

Where 5ab963f2b48d is an image ID, and when you list your images, you'd see that ID in the list e.g.

$ docker image ls --all
REPOSITORY    TAG      IMAGE ID       CREATED        SIZE
<none>        <none>   5ab963f2b48d   7 minutes ago  1.18GB

To run that image (with a terminal) for example, you can simply:

docker run -i -t 5ab963f2b48d /bin/bash

Also see: Run a Docker Image as a Container