Klugscheißer
Klugscheißer

Reputation: 1634

Docker volume mounting shows empty directory with a linux mounted directory

I have a linux volume, dev/sdh, that's mounted at /media/ebs_volume. When I try to mount it into a docker container using -v /media/ebs_volume:/tmp/foo:ro, the mounted directory in /tmp/foo is empty! ls -al /tmp/foo from inside the container is empty but ls -al /media/ebs_volume on the host has stuff in it.

ec2-user@ip-10-0-2-50 ~]$ ls -al /media/ebs_volume/
total 28
drwxr-xrwx 4 root     root      4096 Jan 20 21:34 .
drwxr-xr-x 3 root     root      4096 Jan 29 23:18 ..
drwx------ 2 root     root     16384 Jan 20 21:29 lost+found
drwxrwxr-x 3 ec2-user ec2-user  4096 Jan 20 21:38 stuff
[ec2-user@ip-10-0-2-50 ~]$ docker run -it --rm -v /media/ebs_volume:/tmp/foo:ro nginx ls -al /tmp/foo
total 4
drwxr-xr-x 2 root root 4096 Jan 29 23:18 .
drwxrwxrwt 3 root root   16 Jan 30 17:33 ..

I've tried mounting the volume on the host in different locations, and mounting just a single specific file from the host instead of the entire directory. If I mount something from the home directory that seems to work without any issue.

Anyone have any ideas what might be happening here or best way to debug this?

Possibly useful output with some edits:

[ec2-user@ip-10-0-2-50 ~]$ mount
proc on /proc type proc (rw,relatime)
/dev/xvda1 on / type ext4 (rw,noatime,data=ordered)
/dev/xvdh on /media/ebs_volume type ext4 (rw,relatime,data=ordered)
...                  

[ec2-user@ip-10-0-2-50 ~]$ docker version
Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   7392c3b/1.12.6
 Built:        Fri Jan  6 22:16:21 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   7392c3b/1.12.6
 Built:        Fri Jan  6 22:16:21 2017
 OS/Arch:      linux/amd64

[ec2-user@ip-10-0-2-50 ~]$ cat /proc/version
Linux version 4.4.41-36.55.amzn1.x86_64 (mockbuild@gobi-build-60008) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Wed Jan 18 01:03:26 UTC 2017

[ec2-user@ip-10-0-2-50 ~]$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
1436d3223b27        nginx               "/bin/bash"              47 seconds ago      Up 46 seconds       80/tcp, 443/tcp     happy_leakey

[ec2-user@ip-10-0-2-50 ~]$ docker inspect 1436d3223b27
[
    {
        "Id": "1436d3223b270f78654d26682c469f4d7bdd24766c60f4052cb1a3fb60c86c12",
        "Created": "2017-01-30T17:34:12.430135675Z",
        "Path": "/bin/bash",
        ...
        "Image": "sha256:cc1b614067128cd2f5cdafb258b0a4dd25760f14562bcce516c13f760c3b79c4",
        ...
        "Driver": "devicemapper",
        "MountLabel": "",
        ...
        "HostConfig": {
            "Binds": [
                "/media/ebs_volume:/tmp/foo:ro"
            ],
            "ContainerIDFile": "",
            ...
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            ...
        },
        ...
        "Mounts": [
            {
                "Source": "/media/ebs_volume",
                "Destination": "/tmp/foo",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            ...
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.11.9-1~jessie"
            ],
            "Cmd": [
                "/bin/bash"
            ],
            "Image": "nginx",
            "Volumes": null,
            ...
        },
        ...
    }
]

Upvotes: 5

Views: 971

Answers (1)

Klugscheißer
Klugscheißer

Reputation: 1634

Dirty solution, but I found that restarting the docker service via docker service restart fixed this for me.

I was using ElasticBeanstalk for this and simply added some commands to restart the service on deployment in my .config file eg:

...

commands:
    ...
    02-restart-docker:
        command: service docker restart && docker restart ecs-agent
    ...

See http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html and https://github.com/docker/docker/issues/5489#issuecomment-73427804

Upvotes: 0

Related Questions