CODEWITHSUNDEEP
pythonairflow
luisvenezian
luisvenezian

Reputation: 501

sqlite syntax error with simple task on apache airflow-2.0

i have just installed apache airflow according to the following steps:

  1. pip install "apache-airflow[celery, crypto, postgres, mysql, rabbitmq, redis]" --constraint constraints-3.7.txt
  2. airflow db init
  3. mkdir airflow/dags
  4. Set to False the load_examples variables at airflow.cfg file.
  5. Created a user.

I'm using a Ubuntu 16.04.6 LTS VM and a virtual enviroment with Python 3.7.8 installed.

~$ sqlite3 --version 
3.11.0
>>> import sqlite3
>>> sqlite3.version
'2.6.0'

My dag inside the dags folder is the following:

from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import PythonOperator

from time import sleep
from datetime import datetime

def print_hello():
        sleep(5)
        return 'Hello World'

with DAG('hello_world_dag', description='First DAG', schedule_interval='*/10 * * * *', start_date=datetime(2018, 11, 1), catchup=False) as dag:
        dummy_task      = DummyOperator(task_id='dummy_task', retries=3)
        python_task     = PythonOperator(task_id='python_task', python_callable=print_hello)

        dummy_task >> python_task

At the execution dummy task runs perfectly, otherwise, python task throws this error:

[2020-12-31 12:23:27,493] {taskinstance.py:1396} ERROR - (sqlite3.OperationalError) near ",": syntax error
[SQL: DELETE FROM rendered_task_instance_fields WHERE rendered_task_instance_fields.dag_id = ? AND rendered_task_instance_fields.task_id = ? AND (rendered_task_instance_fields.dag_id, rendered_task_instance_fields.task_id, rendered_task_instance_fields.execution_date) NOT IN (SELECT rendered_task_instance_fields.dag_id, rendered_task_instance_fields.task_id, rendered_task_instance_fields.execution_date 
FROM rendered_task_instance_fields 
WHERE rendered_task_instance_fields.dag_id = ? AND rendered_task_instance_fields.task_id = ? ORDER BY rendered_task_instance_fields.execution_date DESC
 LIMIT ? OFFSET ?)]
[parameters: ('hello_world_dag', 'python_task', 'hello_world_dag', 'python_task', 30, 0)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
Traceback (most recent call last):
  File "/home/vagrant/.sandbox/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
    cursor, statement, parameters, context
  File "/home/vagrant/.sandbox/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: near ",": syntax error

Upvotes: 2

Views: 2079

Answers (1)

Yair hadad
Yair hadad

Reputation: 447

The issue is with the sqlite version,

Databases:" * PostgreSQL: 9.6, 10, 11, 12, 13

  • MySQL: 5.7, 8
  • SQLite: 3.15.0+

ou have 3.11.0

just update (:

you can see it here: https://github.com/apache/airflow/pull/13496/commits/ecd269c776220b67d4cf8f98ca672d77169cb50b

Upvotes: 1

Related Questions