Import into dockerized mariadb on initial build with script

Question

I'm using MariaDB, but I think this could probably apply to MySQL as well.

I have a project that works off of MariaDB, and there is some initial setup for the database that needs to be done to create tables, insert initial data, etc. Based on other answers, I could normally do ADD dump.sql /docker-entrypoint-initdb.d, but I don't have a dump.sql -- instead what I have is a python script that connects to MariaDB directly and creates the tables and data.

I have a docker-compose.yml

version: '3'
services:
  db:
    build: ./db
    ports:
     - "3306:3306"
    container_name: db
    environment:
     - MYSQL_ROOT_PASSWORD=root
  web:
    build: ./web
    command: node /app/src/index.js
    ports:
     - "3000:3000"
    links:
     - db

"Web" is not so important right now since I just want to get db working.

The Dockerfile I've attempted for DB is:

# db/Dockerfile
FROM mariadb:10.3.2

RUN apt-get update && apt-get install -y python python-pip python-dev libmariadbclient-dev
RUN pip install requests mysql-python

ADD pricing_import.py /scripts/

RUN ["/bin/sh", "-c", "python /scripts/pricing_import.py"]

However this doesn't work for various reasons. I've gotten up to the point where pip install mysql-python doesn't compile:

_mysql.c:2005:41: error: 'MYSQL' has no member named 'reconnect'

  if ( reconnect != -1 ) self->connection.reconnect = reconnect;

I think this has to do with the installation of mysql-python.

However before I go down the hole too far, I want to make sure my approach even makes sense since I don't even think the database will be started once I get to the ./pricing_import.py script and since it tries to connect to the database and runs queries, it probably won't work.

Since I can't get the python installation to work on the mariadb container anyway, I was also thinking about creating another docker-compose entry that depends on db and runs the python script on build to do the initial import during docker-compose build.

Are either of these approaches correct, or is there a better way to handle running an initialization script against MariaDB?

Answer 1

We use docker-compose healthcheck with combination of makefile and bash to handle running an initialization script. So your docker-compose.yml would look something like that:

version: '3'
services:
  db:
    build: ./db
    ports:
     - "3306:3306"
    container_name: db
    environment:
     - MYSQL_ROOT_PASSWORD=root
    healthcheck:
     test: mysqlshow --defaults-extra-file=./database/my.cnf
     interval: 5s
     timeout: 60s

Where ./database/my.cnf is the config with credentials:

[client]
host = db
user = root
password = password

Then you can use this health-check.bash script to check the health:

#!/usr/bin/env bash

DATABASE_DOCKER_CONTAINER=$1

# Check on health database server before continuing
function get_service_health {
    # https://gist.github.com/mixja/1ed1314525ba4a04807303dad229f2e1
    docker inspect -f '{{if .State.Running}}{{ .State.Health.Status }}{{end}}' $DATABASE_DOCKER_CONTAINER
}

until [[ $(get_service_health) != starting ]];
    do echo "database: ... Waiting on database Docker Instance to Start";
    sleep 5;
done;

# Instance has finished starting, will be unhealthy until database finishes startup
MYSQL_HEALTH_CHECK_ATTEMPTS=12
until [[ $(get_service_health) == healthy ]]; do
    echo "database: ... Waiting on database service"
    sleep 5
    if [[ $MYSQL_HEALTH_CHECK_ATTEMPTS == 0 ]];
        then echo $DATABASE_DOCKER_CONTAINER ' failed health check (not running or unhealthy)  - ' $(get_service_mysql_health)
        exit 1
    fi;
    MYSQL_HEALTH_CHECK_ATTEMPTS=$((MYSQL_HEALTH_CHECK_ATTEMPTS-1))
done;

echo "Database is healthy"

Finally, you can use makefile to connect all things together. Something like that:

docker-up:
    docker-compose up -d

db-health-check:
    db/health-check.bash db

load-database:
    docker run --rm --interactive --tty --network your_docker_network_name -v `pwd`:/application -w /application your_docker_db_image_name python /application/pricing_import.py

start: docker-up db-health-check load-database

Then start your app with make start.