CODEWITHSUNDEEP
airflow
Gregory Arenius
Gregory Arenius

Reputation: 3224

Run Task on Success or Fail but not on Skipped

Is there a way to run a task if the upstream task succeeded or failed but not if the upstream was skipped?

I am familiar with trigger_rule with the all_done parameter, as mentioned in this other question, but that triggers the task when the upstream has been skipped. I only want the task to fire on the success or failure of the upstream task.

Upvotes: 1

Views: 1759

Answers (1)

Zack
Zack

Reputation: 2466

I don't believe there is a trigger rule for success and failed. What you could do is set up duplicate tasks, one with the trigger rule all_success and one with the trigger rule all_failed. That way, the duplicate task is only triggered if the parents ahead of it fails / succeeds.

I have included code below for you to test for expected results easily.

So, say you have three tasks.

  • task1 is your success / fail
  • task2 is your success only task

  • task3 is your failure only 

    #dags/latest_only_with_trigger.py
import datetime as dt

from airflow.models import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.utils.trigger_rule import TriggerRule


dag = DAG(
dag_id='stackoverflowtest',
schedule_interval=dt.timedelta(minutes=5),
start_date=dt.datetime(2019, 2, 20)
)

task1 = DummyOperator(task_id='task1', dag=dag)
task2 = DummyOperator(task_id='task2', dag=dag,
                      trigger_rule=TriggerRule.all_success)
task3 = DummyOperator(task_id='task3', dag=dag
                      trigger_rule=TriggerRule.all_failed)

###### ORCHESTRATION ###
task2.set_upstream(task1)
task3.set_upstream(task1)

Hope this helps!

Upvotes: 2

Related Questions