CODEWITHSUNDEEP
stored-proceduresazure-data-factoryazure-data-lake
Sandeep T
Sandeep T

Reputation: 441

Event handling in Azure Data Factory

Is there any method available in azure data factory to track the events of a pipeline? I have an event log table and I need to log all the events associated with the pipelines into the same table. Any best practice templates /methods available to achieve the same?

Regards, Sandeep

Upvotes: 2

Views: 635

Answers (1)

Jay Gong
Jay Gong

Reputation: 23782

1.You could use Azure Monitor in ADF and enable diagnostic log and store the logs into Azure Blob Storage.Then analysis logs and transfer them into tables as you want.

2.Another choice,you could use ADF monitor SDK or REST API to retrieve the activities run details by Pipeline Run Id:

List<ActivityRun> activityRuns = client.ActivityRuns.ListByPipelineRun(
resourceGroup, dataFactoryName, runResponse.RunId, DateTime.UtcNow.AddMinutes(-10), DateTime.UtcNow.AddMinutes(10)).ToList(); 
if (pipelineRun.Status == "Succeeded")
    Console.WriteLine(activityRuns.First().Output);
else
    Console.WriteLine(activityRuns.First().Error);

Then you could get all the activities details into tables as you want.

Upvotes: 1

Related Questions