CODEWITHSUNDEEP
pythonairflow
J. Doe
J. Doe

Reputation: 447

In Apache Airflow, can you have a dependency on a previous run of a task?

Let's say A_T means Task A for Today, and A_T-1 means Task A for Yesterday. Is it possible to specify A_T-1 as an operator so that I can have a dependency graph like this:

A_T   >> B_T
A_T-1 >> B_T

Upvotes: 0

Views: 1680

Answers (1)

bruno-uy
bruno-uy

Reputation: 1875

Check official documentation about Cross-DAG Dependencies. Here's an good practice to follow:

When two DAGs have dependency relationships, it is worth considering combining them into a single DAG, which is usually simpler to understand. Airflow also offers better visual representation of dependencies for tasks on the same DAG. However, it is sometimes not practical to put all related tasks on the same DAG.

One example of not having all in the same DAG is the following (from the documentation above):

A task may depend on another task on the same DAG, but for a different execution_date.

I think that could be your case, so if you still need that, you can use the ExternalTaskSensor with the argument execution_delta. Check here for all the details about the parameters of that sensor.

Upvotes: 1

Related Questions