Reputation: 510
I have a BashOperator in my dag which runs a python script. At the end of its run, the python script creates a json as a report. In case of failure, I want to report this json to Slack. I have an on_failure_callback
function which can push a message to Slack. However, I have no elegant way to pass the json value to the dag. Currently, I am saving the json to a file and then reading it from the file in the dag and reporting it. I also tried storing it in an environment variable and getting it in the dag. But is there a more direct way to pass this value to the dag? Preferably, without saving it to a file or an environment variable.
Upvotes: 0
Views: 255
Reputation: 20097
Rather than using on_failure_callback
you should use another task to send your slack message.
And for that you should simply use XComs to push/pull the file. https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html#
Then you can pull the xcom in the task that sends it further to slack.
Upvotes: 1