CODEWITHSUNDEEP
airflow
Vishaal Kalwani
Vishaal Kalwani

Reputation: 740

Airflow Clear automatically triggering tasks?

I am running airflow clear -t task_regex -s 2019-02-23 -e 2019-02-24 dag_id to clear and then re-run a task. However, Airflow is automatically queuing up this task to run, instead of waiting for me to run airflow backfill. Is this a setting I can change? Also, it looks like it queued only some of them to run, and not others. In other words, immediately after clearing the task, half the dates had a green box and the other half had a white box.

Upvotes: 1

Views: 878

Answers (1)

y2k-shubham
y2k-shubham

Reputation: 11607

  • I believe setting catchup_by_default is meant for this very purpose and should be set to False in order to achieve what you want

    Command Line Backfills still work, but the scheduler will not do scheduler catchup if this is False,

  • But then you need not configure it globally since it can be modified on per-DAG basis too using catchup parameter

    however it can be set on a per DAG basis in the DAG definition (catchup)

Upvotes: 1

Related Questions