docker-compose volume question and error --initialize specified but the data directory has files in it. Aborting

Question

I had a spring-boot project that used mysql docker-image so I didn't need to download the mysql benchwork. For other reasons I had to start over so I created a new project that uses the same mysql docker image I previously used. My docker-compose.yml mysql service looks like this

version: "3.7"

services:

  db:
    image: mysql:5.7
    command: --lower_case_table_names=1
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: farming_db
      MYSQL_USER: root
      MYSQL_PASSWORD: root
    restart: always
    volumes:
      - "./database/farming_db/:/var/lib/mysql" #local
      - farming_db:/var/lib/mysql/data #docker
    ports:
      - "3306:3306"
    container_name: farming_mysql
    networks:
      - backend-network

When I run docker-compose up This is the error :

Attaching to farming_mysql, farming_server_springboot_1
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
farming_mysql | 2021-03-18 07:03:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.33-1debian10 started.
farming_mysql | 2021-03-18 07:03:21+00:00 [Note] [Entrypoint]: Initializing database files
farming_mysql | 2021-03-18T07:03:21.058436Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server opti
on (see documentation for more details).
farming_mysql | 2021-03-18T07:03:21.063630Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
farming_mysql | 2021-03-18T07:03:21.063710Z 0 [ERROR] Aborting
farming_mysql |
farming_mysql exited with code 1
springboot_1  |

I understood that my directory is not empty. I am trying to use "./database/farming_db/:/var/lib/mysql" and "farming_db:/var/lib/mysql/data" both as the volume directories. I think the problem is with the latter directory because the prior directory is empty. I'm having a problem deleting the contents in the latter directory because I don't know how to access it.

So this is what I've tried :

1. I deleted all the containers and then deleted all the volumes.docker volume prune but didn't work.

2. I searched that I could do rm -rf /usr/local/var/mysql but I don't know where I can execute this command since the container won't run properly at all.

3. I deleted the mysql image and just ran docker-compose up again. This seems to pull a new mysql image from somewhere? but I still get the same error. I guess volume directory has nothing do with the docker image itself.

4. I deleted the "- farming_db:/var/lib/mysql/data #docker" line from the docker-compose. But the same error is still occuring!

I'm using Windows10.

My question :

1. How can I access the directory? I don't know where to use the rm -rf command.
2. Why does this error still occur even when I erase "- farming_db:/var/lib/mysql/data #docker" from the docker-compose?
3. And also could anyone explain what I am doing? I'm new to docker and I don't really understand these volume problems.

Answer 1

The error message is misleading: Problem resides in permission access by the process to read/write into it. If you create the mysql manually, the dir will be created with the appropriate rights and leave no issue.

Answer 2

Another cause for this problem (happened with me) it's a trivial thing, disk runs out of space. To fix the problem you have to delete the partially created volume/directory after freeing up some space.

Answer 3

If you are using Docker for Mac: Make sure there are no Volumes left that are not removed by docker volume prune.

Had a Volume from a failed docker-compose up attempt, which was not properly removed and it caused the error.

Answer 4

I used docker-compose to run mysql image and encountered the error. I use the following configuration to set volume. - ./mysql/data:/var/lib/mysql/data

Then I changed it to the following and the error was solved. - ./mysql:/var/lib/mysql

Answer 5

As vencedor's answer, it worked for me. If anyone need stay with mysql 5.7, you can add these lines to your db service in docker-compose.yml:

 - /etc/group:/etc/group:ro
 - /etc/passwd:/etc/passwd:ro
user: "1000:1000"

Answer 6

Generally I emptied the volume's data directory and just changed the versions of the MySQL. So in steps:

1. empty volume directory content

2. modify docker-compose.yml mysql version from 5.7 to 5.7.16

Answer 7

Run docker system prune --volumes

This frees up the memory by removing all unused containers. Sometimes, the mentioned issue can occur due to memory limitations

Answer 8

This line indicate that mysql container is storing the data inside a directory database in the same directory than your docker-compose.yml:

    volumes:
      - "./database/farming_db/:/var/lib/mysql" #local

This kind of volume isn't managed by Docker, it's just a directory in your filesystem, this is why docker volume prune doesn't work. I know that, because it starts with a "path" relative or absolute.

The other volume, farming_db, are managed by Docker. I know that because it starts with a simple name. This kinds of volume are managed by Docker and are removed with prune.

So, answering:

1. In the same directory than your docker-compose.yml you can remove that database folder.
2. Because the first volume, the one with /var/lib/mysql still exists. MySQL keeps all files inside this directory and any other child directory are a database.
3. You're just trying to put a container running and docker-compose hides a lot of details.
4. This is just a detail, but MYSQL_USER should be different than root.

You can let Docker manage the entire volume, creating a single volume to hold all data, in this case I named it as mysql_data:

    volumes:
    - mysql_data:/var/lib/mysql

Or, you can explore a bit more the docker run equivalent command to get used with it:

docker run -d --name mysql \
-e MYSQL_ROOT_PASSWORD=root \
-e MYSQL_DATABASE=farming_db \
-e MYSQL_USER=myuser \
-e MYSQL_PASSWORD=mypass \
-v mysql_data:/var/lib/mysql \
-p 3306:3306 \
mysql:5.7