CODEWITHSUNDEEP
mysqlsqliteubuntuairflow
AlphaCR
AlphaCR

Reputation: 847

How do I connect Airflow to SQLite locally?

I'm trying to try out Airflow for the very first time and I'm trying to connect it to a local SQLite database. But I can't seem to get my head around on how to actually do it.

I've read up on Airflow's document, Set my executor to LocalExecutor and set up my sql_alchemy_conn to sqlite:////home/myName/Programs/sqlite3/DatabaseName.db but it doesn't seem to work as it throws an 

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 21, in <module>
    from airflow import configuration
  File "/usr/local/lib/python2.7/dist-packages/airflow/__init__.py", line 35, in <module>
    from airflow import configuration as conf
  File "/usr/local/lib/python2.7/dist-packages/airflow/configuration.py", line 520, in <module>
    conf.read(AIRFLOW_CONFIG)
  File "/usr/local/lib/python2.7/dist-packages/airflow/configuration.py", line 283, in read
    self._validate()
  File "/usr/local/lib/python2.7/dist-packages/airflow/configuration.py", line 169, in _validate
    self.get('core', 'executor')))
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the LocalExecutor

error when I tried to run airflow initdb. I tried to google around and tried vipul sharma's solution found here and set the value of my sql_alchemy_conn to mysql://:@localhost:3306/ but it still doesn't work as it throws an

sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1045, "Access denied for user 'myName'@'localhost' (using password: NO)")

error. I know that the answer should be really simple but I really don't understand how to so I hope you can guide me through on what to do/read.

Upvotes: 5

Views: 13844

Answers (1)

Hasan Jawad
Hasan Jawad

Reputation: 317

Use SequentialExecutor

"This executor will only run one task instance at a time, can be used for debugging. It is also the only executor that can be used with sqlite since sqlite doesn’t support multiple connections." airflow documentation

You just didn't need to change to LocalExecutor, change it back to SequentialExecutor, change sql_alchemy_conn to point to sqlite:////home/myName/Programs/sqlite3/DatabaseName.db and stop airflow services (webserver, scheduler).

Execute airflow initdb then start up the services again.

Hopefully that works.

Upvotes: 6

Related Questions