CODEWITHSUNDEEP
python-3.xairflow
Mansour.M
Mansour.M

Reputation: 510

Pass value from operator to dag

I have a BashOperator in my dag which runs a python script. At the end of its run, the python script creates a json as a report. In case of failure, I want to report this json to Slack. I have an on_failure_callback function which can push a message to Slack. However, I have no elegant way to pass the json value to the dag. Currently, I am saving the json to a file and then reading it from the file in the dag and reporting it. I also tried storing it in an environment variable and getting it in the dag. But is there a more direct way to pass this value to the dag? Preferably, without saving it to a file or an environment variable.

Upvotes: 0

Views: 255

Answers (1)

Jarek Potiuk
Jarek Potiuk

Reputation: 20097

Rather than using on_failure_callback you should use another task to send your slack message.

And for that you should simply use XComs to push/pull the file. https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html#

Then you can pull the xcom in the task that sends it further to slack.

Upvotes: 1

Related Questions