how to correctly remove previous commits of an image in order to build it from the very scratch

Question

After some trying and playing I have a Node.JS project I would like to package. Here is my dockerfile:

FROM node:latest
MAINTAINER me <ma@me.me>
COPY ./backend-codebase /app
RUN /bin/bash -c 'cd /app; npm install'
EXPOSE 80
ENV NODE_ENV=production
CMD [ "sh", "-c", "cd /app; npm run start" ]

and before building it I removed all my images I had previously both pulled and built:

docker rmi $(docker images -q) -f

everything is built and works, but while I was pushing it I realized the size was way too big for my tiny app. So I tried to get an insight into it:

Matteos-MBP-2: matteo$ docker history 98e2a25bffe7
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
98e2a25bffe7        28 seconds ago      /bin/sh -c #(nop)  CMD ["sh" "-c" "cd /app; n   0 B
9d2ee235ec5d        28 seconds ago      /bin/sh -c #(nop)  ENV NODE_ENV=production      0 B
f80a1e12748f        29 seconds ago      /bin/sh -c #(nop)  EXPOSE 80/tcp                0 B
c8353dfcd198        30 seconds ago      /bin/sh -c /bin/bash -c 'cd /app; npm install   0 B
e286e75eb2d9        40 seconds ago      /bin/sh -c #(nop) COPY dir:601eb26de2e2876ab1   18.93 MB
2e81fb2f03ab        44 seconds ago      /bin/sh -c #(nop)  MAINTAINER Me Me   0 B
7c4d899628d5        21 hours ago        /bin/sh -c #(nop)  CMD ["node"]                 0 B
<missing>           21 hours ago        /bin/sh -c curl -SLO "https://nodejs.org/dist   47.39 MB
<missing>           21 hours ago        /bin/sh -c #(nop)  ENV NODE_VERSION=7.4.0       0 B

<missing>           2 weeks ago         /bin/sh -c #(nop)  ENV NPM_CONFIG_LOGLEVEL=in   0 B
<missing>           2 weeks ago         /bin/sh -c set -ex   && for key in     9554F0   108.3 kB
<missing>           2 weeks ago         /bin/sh -c groupadd --gid 1000 node   && user   335.1 kB
<missing>           3 weeks ago         /bin/sh -c apt-get update && apt-get install    322.6 MB
<missing>           3 weeks ago         /bin/sh -c apt-get update && apt-get install    122.6 MB
<missing>           3 weeks ago         /bin/sh -c apt-get update && apt-get install    44.31 MB
<missing>           3 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0 B
<missing>           3 weeks ago         /bin/sh -c #(nop) ADD file:1d214d2782eaccc743   123.1 MB

and here it is: some previous commits are still there, which are related to some old commits that docker produced when I was still playing around with it (see the apt-gets?). When I push it, it tries to push it some way, though it says "missing" in the outpus and I removed them. How can I clean the slate before building it, in order to start afresh and base my next commits on node:latest? Thank you

Answer 1

Your docker rmi likely had errors. The -f needs to come before the image id's you want to remove:

docker rmi -f $(docker images -q)

That said, your history looks perfectly fine. The source images you used to build your container don't get rebuilt, they get pulled, and will have the history from when they were last built, hours and weeks ago.

---

From the comments below, there seems to be some disbelief that all the layers below 7c4d899628d5 are from the node image. Here's a test on my own local environment:

$ docker pull node:latest
latest: Pulling from library/node
75a822cd7888: Already exists
57de64c72267: Already exists
4306be1e8943: Already exists
871436ab7225: Pull complete
0110c26a367a: Pull complete
1f04fe713f1b: Pull complete
723bac39028e: Pull complete
Digest: sha256:08d77f1984cf79739ba7c987636cb871fd69745754200e5891a0c7ee2d9965b0
Status: Downloaded newer image for node:latest

$ docker history node:latest
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
7c4d899628d5        23 hours ago        /bin/sh -c #(nop)  CMD ["node"]                 0 B
<missing>           23 hours ago        /bin/sh -c curl -SLO "https://nodejs.org/dist   47.39 MB
<missing>           23 hours ago        /bin/sh -c #(nop)  ENV NODE_VERSION=7.4.0       0 B
<missing>           2 weeks ago         /bin/sh -c #(nop)  ENV NPM_CONFIG_LOGLEVEL=in   0 B
<missing>           2 weeks ago         /bin/sh -c set -ex   && for key in     9554F0   108.3 kB
<missing>           2 weeks ago         /bin/sh -c groupadd --gid 1000 node   && user   335.1 kB
<missing>           3 weeks ago         /bin/sh -c apt-get update && apt-get install    322.6 MB
<missing>           3 weeks ago         /bin/sh -c apt-get update && apt-get install    122.6 MB
<missing>           3 weeks ago         /bin/sh -c apt-get update && apt-get install    44.31 MB
<missing>           3 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0 B
<missing>           3 weeks ago         /bin/sh -c #(nop) ADD file:1d214d2782eaccc743   123.1 MB

Answer 2

You can find a really detailed answer explaining why you see <missing> in the docker history output for some of the layers, but not others, here:

https://stackoverflow.com/a/35312577/3738611

and a large portion of the information in the answer was found here:

http://www.windsock.io/explaining-docker-image-ids/