CODEWITHSUNDEEP
airflowairflow-scheduler
TheDarkLord
TheDarkLord

Reputation: 351

AirFlow - disable dag after X consecutive fails

I read the API reference and couldnt find anything on it, is that possible?

Upvotes: 3

Views: 1177

Answers (1)

kaxil
kaxil

Reputation: 18824

Currently, there is no such feature that does it out-of-the-box but you can write some custom code in your DAG to get around this. For example, use PythonOperator (you can use MySQL operator if your metadata db is mysql) to get status of the last X runs for the dag. use BranchPythonOperator to see if the number is more than X, if it is then use a BashOperator to run airflow pause dag cli.

You can also just make this a 2-step task by adding logic of PythonOperator in BranchPythonOperator. This is just an idea, you can use a different logic.

Upvotes: 1

Related Questions