CODEWITHSUNDEEP
azureazure-data-factoryazure-logic-appsazure-synapse
AntonyJ
AntonyJ

Reputation: 29

Email errors from all activities in ADF at once

In Azure Data Factory I created a pipeline with several copy data activities. As soon as one of them crashes I want that it continues to the next step and that it sends an e-mail with the error.

The mail error action is correct. When I link it to just one activity and let that one crash on purpose it does send the mail correctly. But when I link it to multiple and it crashes (even the last step) it doesn't send an e-mail.

What am I doing wrong here?

see image

The e-mail step is quite simple, it's set that it triggers an Logic App which sends the actual mail via office365.

Upvotes: 0

Views: 1042

Answers (3)

XchangeVisions
XchangeVisions

Reputation: 1

To send email when a activity fails, you could make a success capture activity that has only 'On success' relations. If that activity is skipped, you know there's a failure before.

Example: simplified pipeline

Upvotes: 0

Nandan
Nandan

Reputation: 4945

In adf, the flows act as AND condition. So based on your above image, an email will be sent only when all activities tagged to the web activity will fail. So to achieve your scenario, you would have to use a combination of on skip, on completion and on failure flows. My blog: https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activities-in-azure-data-factory

Upvotes: 1

Pratik Lad
Pratik Lad

Reputation: 8402

As soon as one of them crashes I want that it continues to the next step and that it sends an e-mail with the error.

All the connections marked "on failure" are the problem. Although what you created makes logical. When an activity receives connections from several dependents, it will not begin to run until all of the dependencies have reported in. for this you can use pipeline dependency called on completion to run next activity even if previous activity is failed.

And another problem in your approach is if one activity is executed successfully and you are trying to fetch error from that in web activity msg It will throw an error

You have to give web activity to each activity to send mail if they fail and give On Completion to all activities It will run all activities irrespective previous activity is failing or not.

enter image description here

Or If your main intension is to get mail if activities failed in pipeline you could create alert on particular pipeline with all activities with Failed activity runs metrics It would send you mail if any activity failed in that particular pipeline

enter image description here

enter image description here

Upvotes: 0

Related Questions