CODEWITHSUNDEEP
pythonairflow
samarth
samarth

Reputation: 4024

Airflow cron expression is not scheduling dag properly

I am exploring Airflow to be used as a cron so that I could use its other features while setting up the cron.

I was testing its functionality by setting cron like "2,3,5,8, * * * *". I was expecting particular dag to be scheduled on minute 2,3,5 and 8 of every hour. However in reality the dag for 2nd minute is executed on 3rd, for 3rd on 5th and for 5th on 8th. And it is not executed for 8th at all. I guess it would be executed for 8th on 2nd minute of the next hour.

Looks like some kind of a bug or not handled case for cron expression in airflow.

Upvotes: 3

Views: 4978

Answers (2)

Mercury
Mercury

Reputation: 350

Yeah, that's because airflow doesn't execute on the interval, but rather after that interval has finished. Since you specify minutes 2, 3, 5, 8 then it will run at the end of those intervals, on 3, 5, 8, and 2. That's explained in the docs here

Upvotes: 1

Antoine Augusti
Antoine Augusti

Reputation: 1608

This is not a bug. I agree it feels weird but this is explained here: https://pythonhosted.org/airflow/scheduler.html

Note that if you run a DAG on a schedule_interval of one day, the run stamped 2016-01-01 will be trigger soon after 2016-01-01T23:59. In other words, the job instance is started once the period it covers has ended.

Upvotes: 1

Related Questions