CODEWITHSUNDEEP
dockerdockerfiledocker-ce
Marius P
Marius P

Reputation: 434

Docker executing cp or mv command in Dockerfile, but changes does not show up in an image

I am quite new to programming and docker. I did come across such a problem, which I couldn't find an answer to.

I created my custom docker image with no problems and then I wanted to use this image as my base for further changes. So I created my Dockerfile, which looks like (see below) and when I use sudo docker build -t my-name . it executes with no problems, however when i run the image (sudo docker run -it my-name /bin/bash) I see no changes in the directory list (directory /root/new_files/ does not exist).

I seem to think this populated somehow from my first build on which my this image is based as I did these commands many times before with no problems.

Any advice, what could have caused Docker executing the program, but changes not seen in the image itself.

My Docker file:

FROM plu_build_1:latest

ENV BASEDIR=/root

WORKDIR /root

RUN cp -a $BASEDIR/TEMPLATE/ $BASEDIR/DEMO/

COPY DEMO/parameters.DEMO $BASEDIR/DEMO/

COPY DEMO/config.DEMO $BASEDIR/DEMO/

ENV PATH="${BASEDIR}/bin:${PATH}"

VOLUME ["/root/DEMO/LOG/"]
CMD ["bash"]

I want to note, that instead of cp -a I tried 'mv' command ... with no luck also seems that creating links with 'ln' does not work.

However if after build I enter image and run the same command inside running image it works fine, that means I can run 'cp -a $BASEDIR/TEMPLATE/ $BASEDIR/DEMO/' and it works.

So, --no-cache didn't help. Below is the base image Dockerfile:

base image Dockerfile:

FROM fedora:25
RUN yum -y update \
  && yum -y install file gcc gcc-gfortran gcc-c++ glibc.i686 libgcc.i686 libpng-devel jasper jasper-devel hostname m4 make perl \ 
  tar tcsh time wget which zlib zlib-devel openssh-clients openssh-server net-tools \
  netcdf-fortran libpng15 iproute-tc tcp_wrappers-libs sendmail procmail psmisc procps-ng mailx  findutils ImageMagick \
  perl-CPAN ncl netcdf libpng libjpeg-turbo which patch vim less bzip2 \
  && yum clean all

RUN yum -y install netcdf-openmpi-devel.x86_64 netcdf-fortran-openmpi-devel.x86_64 netcdf-fortran-openmpi.x86_64 hdf5-openmpi.x86_64 openmpi.x86_64 openmpi-devel.x86_64 \
  && yum clean all

COPY files.tgz /root
COPY files-bin.tgz /root
COPY rings.tgz /root
# extract all and link all files

RUN tar -xvzf files.tgz \
&& tar -xvzf files-bin.tgz \
&& tar -xvzf rings.tgz \
&& rm files*.tgz \
&& rm rings.tgz 

WORKDIR /root/bin
COPY prog-cmake-linux.tar /root/bin
COPY files-cmake-linux.tar /root/bin
RUN tar xf prog-cmake-linux.tar \
 && tar xf files-cmake-linux.tar \
 && rm prog-cmake* \
 && rm files-cmake* \
 && rm -rdf /root/bin/test/ \
 && rm -rdf /root/bin/main/ \
 && rm -rdf /root/bin/*grid/ \
 && mkdir /wrf/netcdf_links \
 && ln -sf /usr/lib64/openmpi/lib /root/netcdf_links/lib \
 && ln -sf /usr/include/openmpi-x86_64 /root/netcdf_links/include 

RUN (echo y;echo o conf prerequisites_policy follow;echo o conf commit) | cpan   && cpan install Proc/Background.pm \
  && ln -s libnetcdff.so.6 /lib64/libnetcdff.so.5 \
  && ln -s libnetcdf.so.11 /lib64/libnetcdf.so.7

RUN echo export LDFLAGS="-lm" >> /etc/bashrc \
 && echo export NETCDF=/root/netcdf_links >> /etc/bashrc \
 && echo export JASPERINC=/usr/include/jasper/ >> /etc/bashrc \
 && echo export JASPERLIB=/usr/lib64/ >> /etc/bashrc \
 && echo export LD_LIBRARY_PATH="/usr/lib64/openmpi/lib" >> /etc/bashrc \
 && echo export PATH="/usr/lib64/openmpi/bin:$PATH" >> /etc/bashrc \
 && echo setenv LDFLAGS "-lm" >> /etc/csh.cshrc \
 && echo setenv NETCDF "/root/netcdf_links" >> /etc/csh.cshrc \
 && echo setenv JASPERINC "/usr/include/jasper/" >> /etc/csh.cshrc \
 && echo setenv JASPERLIB "/usr/lib64/" >> /etc/csh.cshrc \
 && echo setenv LD_LIBRARY_PATH "/usr/lib64/openmpi/lib" >> /etc/csh.cshrc \
 && echo setenv PATH "/usr/lib64/openmpi/bin:$PATH" >> /etc/csh.cshrc \
 && echo export BASEDIR=$BASEDIR >> /etc/bashrc \
 && echo export PATH+=:\$BASEDIR/bin >> /etc/bashrc

ENV LD_LIBRARY_PATH /usr/lib64/openmpi/lib
ENV PATH  /usr/lib64/openmpi/bin:$PATH

# set up ssh configuration
COPY ssh_config /root/.ssh/config
RUN mkdir -p /root/.openmpi
COPY default-mca-params.conf /root/.openmpi/mca-params.conf

RUN mkdir -p /var/run/sshd \
    && ssh-keygen -A \
    && sed -i 's/#PermitRootLogin yes/PermitRootLogin yes/g' /etc/ssh/sshd_config \
    && sed -i 's/#RSAAuthentication yes/RSAAuthentication yes/g' /etc/ssh/sshd_config \
    && sed -i 's/#PubkeyAuthentication yes/PubkeyAuthentication yes/g' /etc/ssh/sshd_config \
    && ssh-keygen -f /root/.ssh/id_rsa -t rsa -N '' \
    && chmod 600 /root/.ssh/config \
    && chmod 700 /root/.ssh \
    && cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
#
WORKDIR /root
VOLUME /root

Upvotes: 3

Views: 32008

Answers (2)

David Maze
David Maze

Reputation: 159733

The immediate cause of your problems is that, once you declare a directory a VOLUME in a Dockerfile, you can never make changes in that directory tree ever again. In particular, since your base image Dockerfile ends with

VOLUME /root

then a couple of steps later

FROM plu_build_1:latest  # inherits that VOLUME
RUN cp -a /root/TEMPLATE/ /root/DEMO/

is a no-op because nothing in /root can ever be changed ever again.

The very short answer here is to never put VOLUME in a Dockerfile at all. It maybe makes sense for things like database servers that have a single directory tree that you almost always want to outlive a single container if you can, but that's an exception, and it has some confusing side effects (like this).

Looking at this Dockerfile more broadly, it looks like a full-blown development environment more than a self-contained reusable image. (It contains two network servers, two compiler stacks and a third interpreted language runtime, and an out-of-the-mainstream interactive shell; it goes out of its way to configure both shells' dotfiles, when many typical Docker paths run no shells at all; it contains an easily extracted ssh key that gives root permissions to something.) You might consider whether a stack built on Vagrant, a full-blown VM, and a more modular configuration management system like Ansible is a better match for what you're trying to build.

Upvotes: 3

sktan
sktan

Reputation: 1259

This is due to the container caching commands when being built. When the dockerfile hasn't changed, it will actually not perform the actions as each command is assumed to have the same result as the previous run (but it will re-run any line that was changed

For example, if cp -a $BASEDIR/TEMPLATE/ $BASEDIR/DEMO/ was changed to cp -ar $BASEDIR/TEMPLATE/ $BASEDIR/DEMO/ for example, it will run that command and every line after as the cache is now invalid.

If you want to update the docker image with your latest changes, you will need to execute docker build with the --no-cache option like so.

 docker build --no-cache my-name .

Upvotes: 0

Related Questions