CODEWITHSUNDEEP
azureazure-synapseazure-synapse-analyticsazure-synapse-pipeline
Scott Stensland
Scott Stensland

Reputation: 28285

How to programmatically retrieve errors from an azure synapse pipeline?

How can I programmatically retrieve error messages from each run of an azure synapse pipeline ? Obviously if this was a normal binary I could redirect stderr and/or stdout by executing the binary from a shell wrapper, however its not that simple here since this is auto launched daily based on a cloud based azure synapse pipeline trigger.

I have many pipelines and wish to achieve this without having to modify the underlying pipelines themselves ( but will if I must )

Any suggestions ? Yes synapse pipelines are a no-code solution however I need access to errors to drive my downstream processing

Upvotes: 0

Views: 231

Answers (1)

Joel Cochran
Joel Cochran

Reputation: 7738

In the Azure SDK, under the Artifacts namespace, you can use the PipelineRunClient class to get information about the run.

Here is some code I use to capture the Status of a run, but it could be used for other properties as well.

protected async Task<PipelineRun> GetRunAsync(string runId)
{
    var run = await Client.GetPipelineRunAsync(runId);
    return run.Value;
}

public async Task<string> GetStatusAsync(string runId)
{
    var run = await GetRunAsync(runId);
    return run.Status;
}

The Client object in this case is a PipelineRunClient. run.Value is a PipelineRun object, which includes the following properties: enter image description here

I have not tested it, but I assume if the Status is Failed, the Message property would contain the error message text.

Upvotes: 0

Related Questions