CODEWITHSUNDEEP
mysqlairflowamazon-rds
kee
kee

Reputation: 11629

How to Support SSL in Airflow MySQLHook (in particular AWS 2019CA)

I have been using MySQLHook happily in my Airflow DAG but now the MySQL server (AWS RDS) will have SSL connection mandatory. My backend engineer told me that in particular AWS 2019 CA should be used. I looked into the MySQLHook documentation and found the following snippet from https://airflow.readthedocs.io/en/stable/_modules/airflow/hooks/mysql_hook.html:

    if conn.extra_dejson.get('ssl', False):
        # SSL parameter for MySQL has to be a dictionary and in case
        # of extra/dejson we can get string if extra is passed via
        # URL parameters
        dejson_ssl = conn.extra_dejson['ssl']
        if isinstance(dejson_ssl, six.string_types):
            dejson_ssl = json.loads(dejson_ssl)
        conn_config['ssl'] = dejson_ssl

It looks like I need to specify some configuration in the form of JSON ("SSL" key) in the extra section of the MySQL connection in Airflow but I couldn't find any examples of this. Can someone enlighten me? Any pointer or an example of such JSON would be very appreciated.

Upvotes: 1

Views: 1165

Answers (1)

joebeeson
joebeeson

Reputation: 4366

Your Connection.extra data should be a JSON string containing a ssl object suitable for passing to the mysql_ssl_set function, according to the "Functions and attributes" section on this page:

This parameter takes a dictionary or mapping, where the keys are parameter names used by the mysql_ssl_set MySQL C API call. If this is set, it initiates an SSL connection to the server; if there is no SSL support in the client, an exception is raised. This must be a keyword parameter.

Presumably something like this would work: {"ssl": {"cert": "PATH TO YOUR PUBLIC CERT FILE ON THE AIRFLOW SERVER"}}

Upvotes: 1

Related Questions