Is there a way to trigger the next task based on previous task run states. Scenario as below: Task1 - First task in m DAG Task2 - Run task2 only when task1 has succeeded Task3 - Run task 3 only when task3 has succeeded Task4 - Run task 4 only when task1 has run more than 10 hours(SLA missed)

Reputation: 61

Airflow trigger tasks only based on previous runs status

Is there a way to trigger the next task based on previous task run states. Scenario as below:

Upvotes: 6

Reputation: 765

You have multiple options here:

Use trigger rules, see trigger-rules on how to use them.
Use on_failure_callback and on_success_callback to define what happens if your task fails/succeeds, see this post or the definitions in the BaseOperator API Reference (see Parameters -> on_failure_callback,on_success_callback).
If you only want the emails to be sent in case of failure or SLA miss and no other task should be executed in that case, define:
default_args = {'email': ['some_email_adress'],'email_on_failure': True"}, airflow will then send an email with the error/sla miss to the defined emails.

Upvotes: 2