General Question about Docker Volumes vs. Bind, sharing and location an OS X

Question

I am still sort of learning some of the features of docker-compose and docker containers in general. I am unclear about the difference between mapping a host folder to a container folder using bind vs. a similar mapping as a volume. Unclear how this relates to declaring a volume also. As an example, I have quite a few containers defined in my docker-compose.yml file, but I started playing with these 2 to work through my problem:

#  MPPS testing server, DICOM

  python_mpps:
 
   build: python_mpps
   image: sdscotti/python_mpps
   depends_on: [mysql_db,pacs-1,pacs-2]
   ports: ["104:11112"]
   volumes:
      - type: bind
        source: ./python_mpps/scripts_log
        target: /scripts
      - type: bind
        source: ./tls
        target: /etc/python/tls
      - type: volume
        source: MWL
        target: /MWL
        volume:
          nocopy: true
   tty: true
   
#  MWL server, REST API

  python_mwl_api:
 
    build: python_mwl_api
    image: sdscotti/python_mwl_api
    depends_on: [mysql_db,pacs-1,pacs-2]
    ports: ["5000:5000"]
    environment:
      PORT: 5000
      FLASK_DEBUG: 1
      FLASK_ENV: development
    volumes:
      - type: bind
        source: ./python_mpps/scripts_log
        target: /scripts
      - type: bind
        source: ./tls
        target: /etc/python/tls
      - type: volume
        source: MWL
        target: /MWL
        volume:
          nocopy: true
    tty: true
    
volumes:
  MWL:

I am using Docker Desktop on a Mac (Catalina) and just the CLI on LINUX, but this pertains to OS X. If I go into the bash shell in both containers and check what is in /etc/python/tls I see what is also on my host in ./tls, which is what I expect to see (e.g.)

docker exec -it 37fabd30c9afe6ca290a3e7f279a7a677061b0fbbc6277650d7055a285fb1ca4 /bin/sh
# cd /etc/python/tls
# ls
USAGE.md  copy-tls-to-docker-volumes.sh  nginx-crt.pem  nginx.cnf
ca.cnf    generate-tls.sh        nginx-key.pem

Same applies for /scripts in each container. Maps to the host as expected, and that is where there are python scripts that are launched on startup and also a log file.

# cd /scripts
# ls
mpps.log  mpps.py
# 

I should say that those scripts folders are only used by that particular container and are not shared with others, whereas the ./tls on the host is read from by many containers, but not written to at all. That is to just copy ssl .crt and .key files to servers since that is a wild card cert.

If I then look at the /MWL folder in a container I do see what I expect to see:

# cd /MWL
# ls
test.wl
# 

but I have no idea where that is on my host system on OS X. I can do:

docker volume inspect orthanc_docker_ris_MWL
[
    {
        "CreatedAt": "2021-07-06T21:08:20Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "orthanc_docker_ris",
            "com.docker.compose.version": "1.29.2",
            "com.docker.compose.volume": "MWL"
        },
        "Mountpoint": "/var/lib/docker/volumes/orthanc_docker_ris_MWL/_data",
        "Name": "orthanc_docker_ris_MWL",
        "Options": null,
        "Scope": "local"
    }
]

which is actually maybe here somewhere:

/Users/xxxx/Library/Containers/com.docker.docker/Data/vms

What I want to do is map the volume to an easily accessible location on my host, like in the root of the docker-compose.yml file. Using bind actually works fine really, but I'm having issues writing to a folder that is bound by more than one container. Uncertain if that is a limitation related to docker itself, or if one of my containers is actually putting a lock on the folder. The goal really is do have a shared bind or mount that can read and written to from multiple containers vs. a volume that can have the same functionality, but I'd like to have a volume be mapped to a folder as described.

I do have 2 instances of "Orthanc PACS" running also, and I tried having them use that volume also, and it looks like that is the case because Docker Desktop shows a total of 4 containers bound, the 2 above or the 2 others.

orthanc_docker_ris_MWL
In use by 4 containers
CREATED
38 minutes ago

That is what I want actually, but I'll have to see if I can write to the volume from multiple containers, and then ideally have it at the root of my docker-compose file rather than in some obscure place on my system in some format unknown to me.

Might mention also that there are few other issues with setup on OS X in that some of the containers (i.e. nginx with php using supervisor) take a long time to start (couple of minutes maybe, although works fine after that), and there is also this issue:

global.stat permission issue mentioned elsewhere. That also occurs on LINUX and looks like there are a number of people with that problem.

I take it Docker Desktop for OS X is not too bad, but much better to use LINUX host. My NGINX - PHP container on LINUX starts immediately, so it must have something to do with the host file system manager.

NOTE:

If I change from a volume to a bind, and delete the volume, it is setup pretty much like I want it, but I'll have to test if more than one container can write to it. I suspect that maybe the PACS is locking the folder somehow.

  - type: bind
    source: ./MWL
    target: /MWL

Answer 1

Docker Desktop for Mac runs dockerd in a Linux virtual machine.

A "volume" is created and stored by dockerd on the Linux virtual machine. The volume is not directly accessible from macos.

The /var/lib/docker/volumes/orthanc_docker_ris_MWL/_data path can be accessed by running a container in the VM's namespace:

$ docker run -it --rm --privileged --pid=host debian nsenter -t1 -m -u -i -n bash
container$ ls -1 /var/lib/docker/volumes/
01143a5cc5474d5052e2bf2ad187e42c419dc8082c8fb6d8bdf705ea94ea4fe1
5754aad4021c67741b589b90bcc3d532a87b619aabb130c1b61dca0f3cf8f0a0
e1372a62523b8667b59fbae31aeda3837e4036c8ea92a6dd7205a22a755e27ab
metadata.db
mongo-data
srv.sqlite

The "bind" mount in Docker Desktop is a bit of a special case. A regular bind mount takes an existing directory and maps it into a container.

Docker Desktop does some magic to "bind" a local mac directory > Linux virtual machine > container. The magic is a user space grpc-fuse plugin that lets the VM/Container see the mac directory. This will always be slower than a volume or a plain Linux host bind mount, particularly when that performance is based on the Linux file cache which can't be used as heavily when changes can come from the mac side.