CODEWITHSUNDEEP
mysqldocker-composeairflow
Christie Chen
Christie Chen

Reputation: 215

Airflow + MySQL in docker compose - Unknown host and Access denied

newbies for airflow and mysql settings here. I am trying to set up airflow in docker-compose with MySQL as backend.

I have my mySQL connection string as sql_alchemy_conn_cmd=mysql://localuser:localpassword@mock_mysql/metastore in airflow.cfg

I have my docker-compose.yml as

version: "3.9"
networks:
  airflow:
services:
  redis_local:
    image: redis:latest
    container_name: redis_server
    ports:
      - 6379:6379
    command: redis-server
    restart: on-failure
    networks:
      - airflow
  mysql_local:
    image: mysql:5.7
    container_name: mysql_local
    environment:
      - MYSQL_ROOT_HOST=%
      - MYSQL_USER=localuser
      - MYSQL_PASSWORD=localpassword
      - MYSQL_ROOT_PASSWORD=localpassword
      - MYSQL_DATABASE=metastore
    volumes:
      - ./script/init.sql:/data/application/init.sql
      - ./dbdata:/var/lib/mysql
    ports:
      - 3306:3306
    restart: on-failure
    healthcheck:
      test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"]
      timeout: 1s
      retries: 10
    command:
      --init-file /data/application/init.sql
      --explicit_defaults_for_timestamp=1
    networks:
      - airflow
  airflow_init_db:
    depends_on:
      - redis_local
      - mysql_local
    container_name: airflow_init_db
    image: apache/airflow:2.0.0-python3.8
    command: airflow initdb

since it's connecting to the metastore database, it will required to set up the database in mysql when the database is starting. i have my init.sql as

CREATE DATABASE IF NOT EXISTS metastore;
CREATE USER 'localuser'@'localhost' IDENTIFIED BY 'localpassword';
GRANT ALL PRIVILEGES ON metastore . * TO 'localuser'@'localhost';
FLUSH PRIVILEGES;

However from the healthcheck, mysql itself was failing with

mysql_local        | 2021-07-06T04:29:04.011037Z 3 [Note] Access denied for user 'root'@'localhost' (using password: NO)

init db and scheduler both failed with

sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2005, "Unknown MySQL server host 'mock_mysql' (-2)")

I wasn't really sure what are missing at this point. really appreciate any help! Thanks

Upvotes: 0

Views: 3165

Answers (1)

Jarek Potiuk
Jarek Potiuk

Reputation: 20077

Two problems.

  1. you should use mysql_local as hostname in your connection string not mock_mysql as this is the name of your MySQL container (and this.is the name defined in DNS by docker-compose)

  2. the healthecheck you use assumes password-less authentication, and likely in default db image of MySQL it is.disabled for security. You should pass user/password as.parameters to your healthecheck command or enable root password-less authentication

Upvotes: 2

Related Questions