CODEWITHSUNDEEP
pythonairflowtaskscheduled-tasksdirected-acyclic-graphs
J. Mini
J. Mini

Reputation: 1610

How can I make a task dependent on another, but also only run at a certain time?

I have two tasks.

  1. T05:30 - I wish for this to run at 05:30 every day.
  2. T08:30 - I wish for this to run at 08:30 every day, but only if T05:30 has succeeded today.

Can this be achieved in Airflow? It appears that dependencies (i.e. run T8:30 only if only if T05:30 has succeeded today) are a task-only concept but schedules (05:30 every day and 08:30 every day) are a DAG-only concept.

I could hack this by checking variables or something external, but I'm hoping that there is an idiomatic solution to this.

Upvotes: 0

Views: 36

Answers (1)

Guilherme Vargas
Guilherme Vargas

Reputation: 26

Maybe you could use a sensor https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/time.html

Think in the dependencies defined ( task_1 >> sensor >> task_2) task_1 will run at 5h30, if it sucessed it will run the sensor (it`ll wait until 8h30) and finally running the task_2.

Upvotes: 0

Related Questions