Reputation: 320
I've got dag_prime and dag_tertiary.
dag_prime
: Scans through a directory and intends to call dag_tertiary
on each one. Currently a PythonOperator. dag_tertiary
: Scans through the directory passed to it and does (possibly time-intensive) calculations on the contents thereof.I can call the secondary one from a system call from the python operator, but i feel like there's got to be a better way. I'd also like to consider queuing the dag_tertiary calls, if there's a simple way to do that. Is there a better way than using system calls?
Thanks!
Upvotes: 13
Views: 33818
Reputation: 551
Use TriggerDagRunOperator
from airflow.operators.dagrun_operator
and pass the other DAG name to triger_dag_id
parameter.
Follow Airflow updated documentation dag_run_operator Airflow Documentation
Upvotes: 0
Reputation: 1649
Use airflow.operators.trigger_dagrun
for calling one DAG from another.
The details can be found in operator trigger_dagrun Airflow documentation.
Following post gives a good example of using this operator: https://www.linkedin.com/pulse/airflow-lesson-1-triggerdagrunoperator-siddharth-anand
Upvotes: 31