CODEWITHSUNDEEP
airflow-schedulerairflow
Adil Blanco
Adil Blanco

Reputation: 666

How to schedule a task at midnight and the other tasks every hour in the same DAG

I wonder if it is possible to use different schedulers in a single DAG, I explain my use case: A task which will be executed every day at midnight to create a file (a new file every day) and the other tasks will be executed every hour to append data into file.

As you notice I have two scheduler_interval

0 0 * * * (Every day at midnight) => create a new file

0 * * * * (Every hour) => append data

Any idea to resolve this problem ?

Upvotes: 0

Views: 480

Answers (1)

Ka Boom
Ka Boom

Reputation: 484

You can use the python branch operator to solve your problem. This python branch operator will check if the current time is midnight then go to create a file task else go to the task to append data into the file. The scheduler should be hourly.

Upvotes: 2

Related Questions