CODEWITHSUNDEEP
dockerdocker-composevirtualbox
Scott Tucker
Scott Tucker

Reputation: 21

Docker in VirtualBox cannot access shared folders

Host OS: macOS Catalina Guest OS: Ubuntu Server 20.04

Hi, I’m really scratching my head here. I am setting up a Docker VM on my file server. I have my main Docker server running Ubuntu Server natively. Unfortunately, there are some Docker containers I need to use that needs the storage on my file server. It’s a 2008 Mac Pro, so Docker for Mac isn’t possible. Docker Toolbox wouldn’t work either. So I just cut out the middle-man and just created a Linux VM manually and running Docker in there. Now, Docker runs great but the containers cannot write to the shared folders I set up in VirtualBox.

I have installed the Guest Additions. I have made my user a member of the vboxsf group (as well as the docker group so I don’t have to use sudo for everything Docker related). I can read and write to the shared folders just fine from the VM machine using nano or whatever tool I need, and Docker itself can create any directory it needs on running docker-compose up -d... it’s just the Docker containers that can’t write to it. It makes no difference if I run docker-compose with sudo or not.

Right now, I’m running Portainer, Bazarr, Radarr, Sonarr, Syslog-ng, and I will be running BackupPC or UrBackup once I get the rest of them being able to write to the shared folders. Most of my containers are running on my dedicated Docker machine, but those that need access to lots of storage will be on this machine.

VirtualBox is running under my user account. Permissions for the folders are 755. I cannot figure out why they cannot write. Any ideas?

Thanks

Edit 8/11/20: I’ve changed up things a bit, Portainer after a couple of hiccups started working. Then I found you could set docker’s API to be monitored from a remote Portainer, so I deleted that one. It seems that Docker itself can use them fine... it’s the containers that are having problems. The bind mapping is done correctly. But the containers cannot see what’s in those folders nor write to them.

At this point, I am working just with Bazarr, Radarr, and Sonarr, since they are so similar or I would have just went to one container. They are running now that I’ve mapped their /config folders to a folder within the VM. But for them to access the storage array, I still have to use shared folders and they cannot seem to cross 2 levels of virtualization. Docker > Ubuntu Server guest OS > macOS Catalina host OS.

Edit 2 by request: Here's my docker-compose.yml file. The commented out containers are that way because I'm trying to work on one thing at a time. The three that aren't commented out are so similar that they may as well be one, so thought it might help me to figure something out if one suddenly started working and the others didn't.

version: "3.7"

networks:
  docker:
    name: DockerBridge
    driver: bridge

services:
############################# NetData
  # netdata:
  #   image: netdata/netdata
  #   container_name: NetData
  #   hostname: azeroth.local
  #   ports:
  #     - 19999:19999
  #   environment:
  #     PUID: ${PUID}
  #     PGID: ${PGID}
  #     TZ: ${TZ}
  #     DOCKER_HOST: proxy:2375
  #   restart: unless-stopped
  #   cap_add:
  #     - SYS_PTRACE
  #   security_opt:
  #     - apparmor:unconfined
  #   volumes:
  #     - /etc/passwd:/host/etc/passwd:ro
  #     - /etc/group:/host/etc/group:ro
  #     - /proc:/host/proc:ro
  #     - /sys:/host/sys:ro
  #     - /etc/os-release:/host/etc/os-release:ro
  #     - ${USERDIR}/netdata/lib:/var/lib/netdata
  #     - ${USERDIR}/netdata/cache:/var/cache/netdata
  #     - ${USERDIR}/netdata/config:/etc/netdata
  #     - ${USERDIR}/nginx/letsencrypt/live:/ssl:ro
  #   network_mode: bridge
  #   depends_on:
  #     - proxy
  # proxy:
  #   image: tecnativa/docker-socket-proxy
  #   container_name: DockerProxy
  #   environment:
  #     PUID: ${PUID}
  #     PGID: ${PGID}
  #     TZ: ${TZ}
  #     CONTAINERS: 1
  #   volumes:
  #     - /var/run/docker.sock:/var/run/docker.sock:ro
  #   network_mode: bridge

############################# Bazarr
  bazarr:
    image: linuxserver/bazarr
    container_name: Bazarr
    environment:
      PUID: ${PUID}
      PGID: ${PGID}
      TZ: ${TZ}
      UMASK_SET: 022 #optional
    volumes:
      - ${USERDIR}/Docker/Bazarr:/config
      - ${MEDIADIR}/Movies:/movies
      - ${MEDIADIR}/TV\ Shows:/tv
    ports:
      - 6767:6767
    restart: unless-stopped
    networks:
      - docker

############################# Radarr
  radarr:
    image: linuxserver/radarr
    container_name: Radarr
    environment:
      PUID: ${PUID}
      PGID: ${PGID}
      TZ: ${TZ}
      UMASK_SET: 022 #optional
    volumes:
      - ${USERDIR}/Docker/Radarr:/config
      - ${MEDIADIR}/Movies:/movies
      - ${MEDIADIR}/Downloads:/downloads
    ports:
      - 7878:7878
    restart: unless-stopped
    networks:
      - docker

############################# Sonarr
  sonarr:
    image: linuxserver/sonarr
    container_name: Sonarr
    environment:
      PUID: ${PUID}
      PGID: ${PGID}
      TZ: ${TZ}
      UMASK_SET: 022 #optional
    volumes:
      - ${USERDIR}/Docker/Sonarr:/config
      - ${MEDIADIR}/TV\ Shows:/tv
      - ${MEDIADIR}/Downloads:/downloads
    ports:
      - 8989:8989
    restart: unless-stopped
    networks:
      - docker

############################# syslog-ng
  # syslog-ng:
  #   image: balabit/syslog-ng:latest
  #   container_name: Syslog-ng
  #   ports:
  #       - 514:514/udp
  #       - 601:601/tcp
  #       - 6514:6514/tcp
  #   environment:
  #       PUID: ${PUID}
  #       PGID: ${PGID}
  #       TZ: ${TZ}
  #   volumes:
  #       - ${HOSTDIR}/Docker/Syslog-ng/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
  #       - ${HOSTDIR}/Docker/Syslog-ng/logs:/var/log
  #   restart: always
  #   networks:
  #     - docker

Here is the .env file, parts of it redacted:

PUID=1000
PGID=1000
TZ=America/New_York
USERDIR=/home/scott/Docker
BACKUPDIR=/media/sf_Backup
HOSTDIR=/media/sf_Host
MEDIADIR=/media/sf_Media
DOMAIN=xxxxxxxx.us
[email protected]

VirtualBox has 3 shared folders.

#1 is going to be for containerizing BackupPC once I get the rest of these working and will be the backup for all my computers. As such, it's not used yet.

#2 is from an earlier setup where I had the /config directories also mapped to a shared folder. I realized I didn't need that, but haven't yet deleted the shared folder in VirtualBox.

#1 is an external eSata ZFS pool. Currently one 6TB drive with the pool set up to easily add more drives as I need them.

#2 is mapped to my macOS home folder on the main SSD.

#3 is a five 4TB hard drive ZFS array with one drive worth of parity, for a capacity of 16TB. The reason for the discrepancy between this and the backup pool is that this array is still mostly empty.

Upvotes: 2

Views: 4208

Answers (1)

agentsmith
agentsmith

Reputation: 1326

I've tested your setup, but I could not find any issues. My setup:

Setup

First: There is no difference during runtime when starting a container with sudo or not. By default, the Unix socket is owned by the user root and other users can only access it using sudo The Docker daemon always runs as the root user. Source: here

  • Host:
    • Microsoft Windows 10 Enterprise
  • Virtual Machine (using Virtual Box 6.1.2)
    • Ubuntu 20.04.1 LTS
  • Docker Installation
    • Docker version 19.03.8, build afacb8b7f0
    • docker-compose version 1.26.0, build d4451659

Docker-Files

I'm invoking in a folder with the following content

developer@appdev:~/tmp$ ls -l
total 16
drwxrwxr-x  2 developer developer 4096 Aug 14 08:59 .
drwxrwx--- 11 developer developer 4096 Jul 29 09:50 ..
-rw-rw-r--  1 developer developer  261 Aug 14 08:23 docker-compose.yaml
-rw-rw-r--  1 developer developer  111 Aug 14 08:17 .env

The docker-compose.yaml file

sonarr:
    image: linuxserver/sonarr
    container_name: Sonarr
    environment:
      PUID: ${PUID}
      PGID: ${PGID}
      UMASK_SET: 022 #optional
    volumes:
      - ${HOSTDIR}/testdir:/Documents
    ports:
      - 8989:8989
    restart: unless-stopped

The .env file

PUID=1000
PGID=1000
TZ=America/New_York
HOSTDIR=/media/sf_Documents
DOMAIN=xxxxxxxx.us
[email protected]

Virtual Box's shared folders

My shared folder is mapped to /media/sf_Documents

developer@appdev:~$ ls -l /media/
total 8
drwxrwx---  1 root vboxsf 4096 Aug 12 09:01 sf_Documents

The folder sf_Dockument contains a folder testdir which has been created by the Host (Windows)

Testing

For testing I stared up the docker-file using

docker-compose up

and let it run. To enter the container with bash interactively:

docker exec -it Sonarr bash

Tried adding a file using cd Documents/, touch test and mkdir mytestdir . An ls -l outputs the following

total 4
drwxrwx--- 1 root  998    0 Aug 14 07:08 .
drwxr-xr-x 1 root root 4096 Aug 14 06:23 ..
drwxrwx--- 1 root  998    0 Aug 14 07:08 mytestdir
-rwxrwx--- 1 root  998    0 Aug 14 06:24 test

A powershell dir invoke outputs the files as expected.

PS D:\UserData\***\Documents\testdir> dir


    Verzeichnis: D:\UserData\***\Documents\testdir


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       14.08.2020     09:08                mytestdir
-a----       14.08.2020     08:24              0 test

My suggestions to narrow down the problem

Since I was not able to reproduce your issues, I want to give you some suggestions I would try:

  • Mount another folder which is not a shared folder (e. g. /home/) and try if you can write to it from your docker-container.

Since I don't use Mac OS as a host system, I can’t verify this. Maybe there are permission issues with your host.

  • Is writing to your shared folder from your VM (not docker!) possible? Try adding a file or a folder to the shared folders and check if it is possible to write to it.
  • Check if Virtual Box can write or read the folders you mapped to the VM. Check the permissions on the folder you want to map to your VM.
  • Check in the Virtual-Box Shared-Folders-Settings if you have ticked Read Only by mistake.

Upvotes: 2

Related Questions