CODEWITHSUNDEEP
airflowgoogle-cloud-composer
franco pina
franco pina

Reputation: 333

Execution of next task in case of failure- airflow

1.4 with composer 2.0.

I have a DAG that runs multiple tasks, the problem I have is that when one task fails, it runs the next one anyway.

According to the airflow documentation this should not be the case, but rather terminate the execution of the DAG after the task fails. Tasks are dependent, so if one fails, the next will fail. I want that in case a task fails, the execution of the DAG will be terminated.

enter image description here

default_args = {
    'owner': owner,
    'start_date': datetime.datetime(2021, 12, 28 ,15 ,0 ,0 ), #2021-08-08 10:00:00 UTC-0
    'email': email,
    'email_on_failure': True,
    'retries': 0,  # Retry once before failing the task.
    #'on_failure_callback': incident_pg, #ejecuta funcion en caso de que la tarea falle 
}
with DAG(dag_id=inst_dag_id,
                default_args = default_args,
                catchup = True,  
                max_active_runs = 5,
                #schedule_interval = None) as dag: #ejecucion manual
                schedule_interval = "0 15 * * *" ) as dag:

Upvotes: 0

Views: 1521

Answers (1)

Tameem
Tameem

Reputation: 418

Looks like the issue is with task definition. It would be more transparent to have the task code in your question. From what you have in the question, there is no 'trigger_rule' parameter defined and from apache-airflow's definition of BaseOperator, the trigger_rule by default is all_success which means that all the upstream tasks are to be successful before downstream tasks can execute.

Check if you have the trigger rule on the task delete_bq_table as all_done if so, remove that or change it to all_success

Upvotes: 1

Related Questions