CODEWITHSUNDEEP
airflow
David Batista
David Batista

Reputation: 3134

Airflow: changing the crontab time for a DAG in Airflow

I have a DAG that has been running everyday at 3:00, it ran OK for the past few weeks.

I've updated the date to run now at 7:00, but apparently for the last 2 days it didn't run. I can see the tasks for those two days with the status 'running' (in green), but no command is triggered.

Does one needs to do something more to change de running time of a DAG ?

I know that in the past one way to solve this was to clean in the meta-database the tasks for this DAG, and update the start_date, but I would rather avoid doing this again.

Anyone has a suggestion?

Upvotes: 20

Views: 28285

Answers (4)

Chris Feng
Chris Feng

Reputation: 199

You can use the same dag. After modifying schedule_interval, you need to mark the previous job as succeeded via airflow backfill -m command.

Upvotes: 3

AnandViswanathan89
AnandViswanathan89

Reputation: 119

David,
1. You can also delete the dag via Experimental REST API. deleting a DAG
2. Change the desired start_date.
3. And add the same DAG back.

Upvotes: 3

p.magalhaes
p.magalhaes

Reputation: 8374

To schedule a dag, Airflow just looks for the last execution date and sum the schedule interval. If this time has expired it will run the dag. You cannot simple update the start date. A simple way to do this is edit your start date and schedule interval, rename your dag (e.g. xxxx_v2.py) and redeploy it.

Upvotes: 17

Conor
Conor

Reputation: 1526

An alternative solution to renaming the DAG is to edit the execution_date of all prior task instances and DAG runs of the DAG in the database. The tables to alter are task_instance and dag_run respectively.

One of the downsides of this approach is that you will lose the ability to browse logs of completed tasks through the webserver.

Upvotes: 5

Related Questions