How do I populate a volume in a docker-compose.yaml

Question

I am starting to write my first docker-compose.yml file to set a a combination of services that make up my application (all node-js). One of the services (web-server - bespoke, not express) has both a large set of modules it needs and an even larger set of bower_components.

In order to provide separation of concerns, and so I can control the versioning more closely I want to create two named volumes which hold the node_modules and bower_components, and mount those volumes on to the relevant directories of the web-server service.

The question that is confusing me is how do I get these two volumes populated on service startup. There are two reasons for my confusion:-

1. The behaviour of docker-compose with the -d flag versus the docker run command with the -d flag - the web service obviously needs to keep running (and indeed needs to be restarted if it fails) whereas the container that might populate one or other of the volumes is a run once as the whole application is brought up with docker-compose up command. Can I control this?
2. A running service and the build commands of that service. Could I actually use a Dockerfiles to run npm install and bower install. In particular, if I change the source code of the web application, but the modules and bower_components don't change, will this build step be instantaneous because of a cached result?

I have been unable to find examples of this sort of behaviour so I am puzzled as to how to go about doing it. Can someone help.

Answer 1

I did sommething like that without bower but with nodeJS tools like Sass, Hall, live reload, jasmine... I used npm for all installation inside the npm project (not global install) For that, the official node image is quiet well, I only have to set the PATH to the app/node_modules/.bin. So my Dockerfile look like this (very simple) :

FROM node:7.5
ENV PATH /usr/src/app/node_modules/.bin/:$PATH

My docker-compose.yml file is :

version: '2'
services:
  mydata:
  image: busybox
  stdin_open: true
  volumes:
    - .:/usr/src/app
node:
  build: .
  image: mynodecanvassvg
  working_dir: /usr/src/app
  stdin_open: true
  volumes_from:
    - mydata
sass:
  depends_on:
    - node
  image: mynodecanvassvg
  working_dir: /usr/src/app
  volumes_from:
    - mydata
  #entrypoint: "node-sass -w -r -o public/css src/scss"
  stdin_open: true
jasmine:
  depends_on:
    - node
  image: mynodecanvassvg
  working_dir: /usr/src/app
  volumes_from:
    - mydata
  #entrypoint: "jasmine-node --coffee --autoTest tests/coffee"
  stdin_open: true
live:
  depends_on:
    - node
  image: mynodecanvassvg
  working_dir: /usr/src/app
  volumes_from:
    - mydata
  ports:
    - 35729:35729
  stdin_open: true

I have only some trouble with entrypoints that all needs a terminal to display result while working. So, I use the stdin_open: true to keep the container active and then I use the docker exec -it on each containers to get running each watch services.

And of course I launch the docker-compose with the -d to keep it alive as daemon. Next you have to put your npm package.json on your app folder (next to Dockerfile and docker-compose.yml) and launch a npm update to load and install the modules.

Answer 2

I'll start with the standard way first

2. Dockerfile

Using a Dockerfile avoids trying to work out how to setup docker-compose service dependencies or external build scripts to get volumes populated and working before a docker-compose up.

A Dockerfile can be setup so only changes to the bower.json and package.json will trigger a reinstall of node_modules or bower_components.

The command that installs first will, at some point, have to invalidate the second commands cache though so the order you put them in matters. Which ever updates the least, or is significantly slower should go first. You may need to manually install bower globally if you want to run the bower command first.

If you are worried about NPM versioning, look at using yarn and a yarn.lock file. Yarn will speed things up a little bit too. Bower can just set specific versions as it doesn't have the same sub module versioning issues NPM does.

File Dockerfile

FROM mhart/alpine-node:6.9.5

RUN npm install bower -g

WORKDIR /app

COPY package.json /app/
RUN npm install --production

COPY bower.json /app/
RUN bower install

COPY / /app/
CMD ["node", "server.js"]

File .dockerignore

node_modules/
bower_components/

This is all supported in a docker-compose build: stanza

1. Docker Compose + Volumes

The easiest/quickest way to populate a volume is by defining a VOLUME in the Dockerfile after the directory has been populated in the image. This will work via compose. I'd question the point of using a volume when the image already has the required content though...

Any other methods of population will require some custom build scripts outside of compose. One option would be to docker run a container with the required volume attached and populate it with npm/bower install.

docker run \
  --volume myapp_bower_components:/bower_components \
  --volume bower.json:/bower.json \
  mhart/alpine-node:6.9.5 \
  npm install bower -g && bower install

and

docker run \
  --volume myapp_mode_modules:/node_modules \
  --volume package.json:/package.json \
  mhart/alpine-node:6.9.5 \
  npm install --production

Then you will be able to mount the populated volume on your app container

docker run \
  --volume myapp_bower_components:/bower_components \
  --volume myapp_node_modules:/node_modules \
  --port 3000:3000
  my/app

You'd probably need to come up with some sort of versioning scheme for the volume name as well so you could roll back. Sounds like a lot of effort for something an image already does for you.

Or possibly look at rocker, which provides an alternate docker build system and lets you do all the things Docker devs rail against, like mounting a directory during a build. Again this is stepping outside of what Docker Compose supports.