How to save output error in blob storage using Azure Data Factory

Question

I have a web activity in Azure data factory pipeline and I need to save the requests body error in a file (txt,csv ...) inside blob storage, is there any way to do this? Already try the azure function, but no success.

Answer 1

After the failure of the web activity, use a copy activity to achieve your requirement.

First create a source dummy.csv file with a single column and single row like below.

Create a csv dataset for the above file and give it to the copy activity source. The copy activity should be after the web activity failure.

In the copy activity source, create two additional columns, one for the error_type and another is for the error_description and give the dynamic expressions like @activity('Web1').output.error and @activity('Web1').output.error_description for those like below.

Create another csv dataset and give your target location with required filename. The copy activity will generate the target file after the pipeline run.

Now, go to copy activity mapping and click on import schema and it will ask the values for the above expressions. Provide sample values for it (Dummy values to set the mapping, these values won't be copied to the target file) like below.

Click on ok and it will give the mapping. In this mapping, remove the dummy_column like below.

Now, Debug the pipeline and pipeline will generate a target file with the error_type and error_description of the web activity.