CODEWITHSUNDEEP
pythonairflow
BugCatcherJoe
BugCatcherJoe

Reputation: 497

How to kill airflow dag run that has already been marked as failed?

I am very new to airflow so please excuse the noobie question.

Like the title says I have an airflow dag that has been marked as failed but still gets ran by airflow scheduler. I see it in my logs. I can kill a process but the scheduler keeps respawning it in another process. How can I stop this?

Upvotes: 0

Views: 1537

Answers (1)

raphael
raphael

Reputation: 2823

When you kill a process you are only killing a scheduled instance of a task/dag, the scheduler will continue to create new instances based on the schedule you have provided.

For example, this example dag has schedule_interval=None.

You may have also set Catchup=True and set your start_date in the past, so the scheduler is backfilling all the past dates.

Upvotes: 1

Related Questions