CODEWITHSUNDEEP
pythontestingairflow
Bruno Faria
Bruno Faria

Reputation: 549

How to test Apache Airflow tasks that uses XCom

I'm trying to figure out a way to test a DAG where I have a couple of tasks communicating using XCom.

Since the console command only allow me to run tasks from a DAG, is there a way to test the communication without having to run the DAG via the UI?

Thanks

Upvotes: 13

Views: 6537

Answers (1)

Bruno Faria
Bruno Faria

Reputation: 549

Here's a way that worked for me.

Even though the Airflow web page states that the test command does not generate or keeps any state, running the airflow test command in sequence worked.

Basically you do:

Airflow test my_dag task1 date
Airflow test my_dag task2 date

where task1 returns the value or uses xcom_push method to send the value and then task2 run xcom_pull to get the value and move on.

It worked for me. Please comment if you have different approaches or other questions.

Upvotes: 10

Related Questions