CODEWITHSUNDEEP
azureazure-data-factoryazure-log-analyticsazure-monitoring
inspiredd
inspiredd

Reputation: 217

Get Azure Data Factory logs

I need to retrieve azure data factory pipelines execution logs. I've tried with Web Acticity by using the following request:

GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}?api-version=2018-06-01

Unfortunatelly I have the following error:

Invoking Web Activity failed with HttpStatusCode - 'NotFound', message - 'The requested resource does not exist on the server. Please verify the request server and retry'

Should I set some additionall configuration to be able to retrieve these logs?

Upvotes: 0

Views: 3345

Answers (1)

KarthikBhyresh-MT
KarthikBhyresh-MT

Reputation: 5034

You can get a list of pipeline runs using Pipeline Runs - Query By Factory

Next, you can Get a pipeline run by its run ID.

GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelineruns/{runId}?api-version=2018-06-01

Here is a sample URL:

https://management.azure.com/subscriptions/b83c1ed3-XXXX-XXX-XXXXX-2n83a074t23f/resourceGroups/resource-grp/providers/Microsoft.DataFactory/factories/ktestadf/pipelineruns/0bdaba11-47b7-4885-9796-5801b4bb856a?api-version=2018-06-01

If you are constructing URL dynamically, using Pipeline RunID system variable, you can using string interpolation method. Notice @{pipeline().RunId} in place of {runId}.

https://management.azure.com/subscriptions/b83c1ed3-XXXX-XXXX-XXXXX-2n83a074t23f/resourceGroups/resource-grp/providers/Microsoft.DataFactory/factories/ktestadf/pipelineruns/@{pipeline().RunId}?api-version=2018-06-01

Note: You would have to Trigger run and not debug, since this will create pipeline. Make sure you have published all before trigering run, debug can take the changes but pipeline run needs the changes be published.

enter image description here

And here is a simple WebActivity setup:

enter image description here

Input

{
    "url": " https://management.azure.com/subscriptions/b83c1td3-XXXX-XXXX-XXXXX-2b83a074c13f/resourceGroups/myrg/providers/Microsoft.DataFactory/factories/ktestadf/pipelineruns/0bdaba11-47b7-4885-9796-5801b4bb856a?api-version=2018-06-01 ",
    "method": "GET",
    "headers": {
        "Content-Type": "application/json"
    },
    "authentication": {
        "type": "MSI",
        "resource": " https://management.azure.com/  "
    }
}

enter image description here

You can get a pipeline run ID from here manually to test.

enter image description here

I did repro before posting this and the only reason this error pops is cause any value you provided is wrong or does not already exist (in case of pipeline run id)

enter image description here

Upvotes: 2

Related Questions