Shared volume between docker-compose files

Question

I want to shared a volume between 2 docker-compose files. There are 2 interconnected apps and I need to create a symlink between them. I tried using named volumes and the external feature.

On the first container, I can see the contents of the /var/www/s1 folder, but the /var/www/s2 folder is empty, while on the second container I can see the contents of the /var/www/s2 folder, but the /var/www/s1 folder seems empty. Since I can't see the contents of the folder created by the other app in /var/www, I can't do a symlink.

I made some dummy docker-compose files to try to expose the problem in an easier way.

In /var/www/s1 there should be a "magazine.txt" file, while in /var/www/s2 there should be a "paper.txt" file.

The first docker-compose file looks like this:

services:
  nginx:
    image: nginx
    container_name: nginx
    volumes:
      - ../:/var/www/s1
      - shared-s:/var/www
volumes:
  shared-s:
    name: shared-s

The second docker-compose file looks like this:

version: '3.8'
services:
  php:
    image: php
    container_name: php
    command: tail -F /var/www/s2/paper.txt
    volumes:
      - ../:/var/www/s2
      - shared-s:/var/www
volumes:
  shared-s:
    external:
      name: shared-s

$ docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                              NAMES
80b83a60a0e5        php                   "docker-php-entrypoi…"   2 seconds ago       Up 1 second                                            php
05addf1fc24e        nginx                 "/docker-entrypoint.…"   9 seconds ago       Up 8 seconds        80/tcp                             nginx
8c596d21cf7b        portainer/portainer   "/portainer"             2 hours ago         Up About a minute   9000/tcp, 0.0.0.0:9001->9001/tcp   portainer

$ docker exec -it 05addf1fc24e sh
# cd /var/www
# ls
s1  s2
# cd s1
# ls
docker  magazine.txt
# cd ..
# cd s2
# ls
# exit
$ docker exec -it 80b83a60a0e5 sh
# cd /var/www
# ls
s1  s2
# cd s1
# ls
# cd ..
# cd s2
# ls
docker  paper.txt
# exit

Answer 1

At a mechanical level, volumes and bind mounts don't "nest" in the way you're suggesting. The named volume shared-s will wind up containing only empty directories s1 and s2, but none of the content from either host directory.

What happens is something like this:

1. Docker starts (say) the nginx container first. It sorts the volumes: mounts on that container from shortest to longest.
2. Since the shared-s volume is empty, the content from the nginx base image in /var/www is copied to the volume; then the volume is mounted on /var/www in the container.
3. Docker creates the mount point /var/www/s1 (in the volume), then bind-mounts the host directory there (without modifying the volume at all).
4. Docker starts the php container and sorts its volumes:.
5. Since the shared-s volume is not empty, Docker just mounts it into the container, hiding any content that might have been in /var/www in the image.
6. Docker creates the mount point /var/www/s2 (in the volume), then bind-mounts the host directory there (without modifying the volume at all).

You'll notice a couple of problems with this sequence. Other mounted volumes' content never gets copied into the "shared" volume, which breaks the file sharing you're attempting here. Whichever container starts up first copies content from its image into the shared volume, and anything in that image in the other container gets lost. For that matter, if there is an update in the base image, Docker will ignore that in favor of the (old) content that's in the shared volume.

I'd suggest avoiding volumes here entirely. Build a separate image for each container, COPYing your application code into it. If you can use a static file server in the backend application, that will be much easier than trying to copy files from one container to the other. If that's not avoidable, you can use the COPY --from=image syntax that's normally used with multi-stage builds to also copy content from one built image to another.