CODEWITHSUNDEEP
airflowairflow-scheduler
ninjaturtle
ninjaturtle

Reputation: 541

Airflow backfill new task added to dag

Lets say today is 2017-10-20. I have an existing dag which is successful till today. I need to add a task with a start_date of 2017-10-01. How to make the scheduler trigger task from 2017-10-01 to 2017-10-20 automatically ?

Upvotes: 10

Views: 7779

Answers (2)

Javier Montón
Javier Montón

Reputation: 5716

If you don't mind executing the whole DAG again, you can remove it from the Web UI and it will appear again with status Off. If you enable it again, it will run from the beginning, including the new tasks.

Upvotes: -1

Daniel Huang
Daniel Huang

Reputation: 6548

You can use the backfill command line tool.

airflow backfill your_dag_id -s 2017-10-01 -e 2017-10-20 -t task_name_regex

This is assuming there is already a DAG run for dates beginning from 2017-10-01. If that's not the case, make sure the DAG's start date is 2017-10-01 or earlier and that catchup is enabled.

Upvotes: 7

Related Questions