Can I tell Docker where in my filesystem to put read-only and read-write parts

Question

I have developed a nodejs application which provides a https (http/2) interface on the one side and talks to an SQL Server database, an SMTP mail sender, and an http based postcode lookup service on the other. In essence its a web based single page application.

This runs on a small raspberry pi/2 sitting on a company lan, talking to the database on a windows machine on the lan. The other services are external. It is important that the application remains private.

This is an early release of this application, so will be subject to some change. The most complex of which is going to be how to upgrade to newer versions of node without long downtimes.

Yesterday I came across Docker, and more specifically an article about how to set up Docker on a raspberry pi. It excited me that this might be the solution to updating my application rapidly. If I package node and my application into a Docker container I can pre-build containers for the latest version of node and my application, test them and the install them very rapidly indeed.

Currently the SD card for the Raspberry Pi's filesystem is configured into 4 partitions. 1) is the boot partition, 2) is the root of the filesystem. These two are both read-only, 3) is the partition for the pi user's home and 4) is /var These latter two are read-write.

The reason for this is that the software once installed is all read only except for the logs (which go into /var). To protect against SD Card Corruption on Power Off. I do have a user who has a home directory in the read only area, and its set so that logon turns the filesystem rw until he logs off again. This user is where I run the commands to install the latest production release. Some testing and development can take place in the pi user, which is why the area is read-write.

I have just installed Docker on my desktop (amd64) machine running Debian. As far as I can tell all of Docker files sit somewhere under /var/lib/docker. This includes all the files that make up the loaded images. Although there is a /etc/docker directory the only configuration in there appears to be a key.

So my question is, Is there a way to configure Docker so that the container images can actually sit in the area that switches between read-write when I log into the account there and become read-only again when I log out, but the parts of the containers that get written to (maybe none - see next paragraph) sit is /var.

My application actually writes its logs to stdout and stderr. At the moment they are being managed by PM2, which restarts them when they exit, but also writes the logs to a pair of log files which it rotates. Unfortunately I have had instances where the application has failed and PM2 hasn't restarted it

I suspect Docker may also be setup to automatically restart a failing container, and that I could also set it up so that the contents stdout and stderr get redirected into a file outside of the Docker container. This would make it easier to view them later. Is this possible? Indeed if it is possible is it good practice, or would I do better to continue to use pm2 inside the Docker container

Answer 1

I have just installed Docker on my desktop (amd64) machine running Debian. As far as I can tell all of Docker files sit somewhere under /var/lib/docker. This includes all the files that make up the loaded images. Although there is a /etc/docker directory the only configuration in there appears to be a key.

So my question is, Is there a way to configure Docker so that the container images can actually sit in the area that switches between read-write when I log into the account there and become read-only again when I log out, but the parts of the containers that get written to (maybe none - see next paragraph) sit is /var.

You can change the Docker root directory from /var/lib/docker to any other directory by setting the daemon's --graph (or short -g) flag. When using systemd, you can accomplish this using a simple drop-in (for example, in /etc/systemd/system/docker.service.d):

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --graph=/path/to/readonly/dir

The exact procedure may vary depending on your Docker version and Linux distribution. See the documentation for more information.

However, this will affect both the image and container file system storage. Having one read-only and the other writeable, is not possible this way.

My application actually writes its logs to stdout and stderr. At the moment they are being managed by PM2, which restarts them when they exit, but also writes the logs to a pair of log files which it rotates. Unfortunately I have had instances where the application has failed and PM2 hasn't restarted it

I suspect Docker may also be setup to automatically restart a failing container, and that I could also set it up so that the contents stdout and stderr get redirected into a file outside of the Docker container. This would make it easier to view them later. Is this possible? Indeed if it is possible is it good practice, or would I do better to continue to use pm2 inside the Docker container

You can start your containers with --restart=on-failure (see more on restart policies). This will cause the Docker engine to automatically restart the container if it exists with a non-zero exit code. Alternatively, use --restart=always to have the container restarted regardless of exit code. Personally, I would recommend against using PM2 inside Docker for a simple restart-on-failure, unless you need any of PM2's additional features.

Regarding logging: By default, Docker will capture all STDOUT and STDERR output of the container's main process and store them in its own logfiles (by default, in /var/lib/docker/containers/<CONTAINER-ID>/<CONTAINER-ID>-json.log). You can view them using docker logs <CONTAINER-NAME>.

If this is not what you want, you can change the logging mechanism by setting the --log-driver flag when starting a container (for example, to --log-driver=syslog). Again, see the documentation for more information.