CODEWITHSUNDEEP
azureazure-data-factory
Blue Clouds
Blue Clouds

Reputation: 8161

Standard way to send ADF email on error per activity

I have several activities piped together horizontally and vertically as given in the diagram. When an error happens then I need to send the activity name and it's error details in an email. The only ugly way I can think of is to attach an error line from each activity to an email activity(per activity). Is there a better way(sort of try catch) or can this arranged with single "error email" activity?

enter image description here

Upvotes: 0

Views: 3223

Answers (2)

Sarang Kulkarni
Sarang Kulkarni

Reputation: 367

Create one master pipeline execute this pipeline inside it and add send mail activity on failure

@activity('activityname').error.message

Upvotes: 0

Trent Tamura
Trent Tamura

Reputation: 1145

One solution that I like is to query the ADF Activity Run API to get a list of the failed activities for that Pipeline Run. You can do this at the end of the Pipeline Run to send in a single email all the errors for that Pipeline, or you can send an email per failed activity. Your choice.

You can use examples from this StackOverFlow to learn how to use the API:

How to get output parameter from Executed Pipeline in ADF?

Here is the API URL for querying ADF Activity Runs:

https://management.azure.com/subscriptions/@{pipeline().parameters.SubscriptionId}/resourceGroups/@{pipeline().parameters.ResourceGroupName}/providers/Microsoft.DataFactory/factories/@{pipeline().DataFactory}/pipelineruns/@{pipeline().runId}/queryActivityruns?api-version=2018-06-01

The Body:

{
  "lastUpdatedAfter": "2018-06-16T00:36:44.3345758Z",
  "lastUpdatedBefore": "@{utcnow()}",
  "filters": [
    {
      "operand": "Status",
      "operator": "Equals",
      "values": [
        "Failed"
      ]
    }
  ]
}

Using this API call you can get a JSON response for all failed activities in your pipeline, you can then use this to send an email however you like (my personal preference is a SendGrid API Call) if you need help with actually sending the email let me know, but this should get you the data you need in a single activity.

EDIT:

I just realized those are separate Pipeline executions not activities, so assuming you want to know what pipelines failed and their error messages, you can instead use a Query Pipeline API, which is still provided in the StackOverflow link. You can then get a list of the failed pipelines and filter by the invoked pipeline ID (which is just the parent Pipeline RunID):

Pipeline Invoked by

Upvotes: 1

Related Questions