CODEWITHSUNDEEP
airflow
Programmer120
Programmer120

Reputation: 2592

How to exit with error from script to Airflow?

Say I'm running:

t = BashOperator(
    task_id='import',
    bash_command="""python3 script.py '{{ next_execution_date }}' """,
    dag=dag)

And for some reason I want the script to exit with error and indicate airflow that he should retry this task.

I tried to use os._exit(1) but Airflow mark the task as success. I know there is:

from airflow.exceptions import AirflowException
raise AirflowException("error msg")

But this is more for functions written in a DAG. My script is independent and sometimes we run it alone regardless of airflow.

Also the script is Python3 while Airflow is running under Python 2.7 It's seems excessive to install Airflow on Python3 just for error handling.

Is that any other solution?

Upvotes: 0

Views: 3767

Answers (1)

kaxil
kaxil

Reputation: 18884

Add || exit 1 at the end of your Bash command:

bash_command="""python3 script.py '{{ next_execution_date }}' || exit 1 """

More information: https://unix.stackexchange.com/a/309344

Upvotes: 2

Related Questions