CODEWITHSUNDEEP
azureazure-data-factory
Johannes Vink
Johannes Vink

Reputation: 173

Azure Data Factory v2: Activity execute pipeline output

Is there a way to reference the output of an executed pipeline in the activity "Execute pipeline"?

I.e.: master pipeline executes 2 pipelines in sequence. The first pipeline generates an own created run_id that needs to be forwarded as a parameter to the second pipeline.

I've read the documentation and checked that the master pipeline log the output of the first pipeline, but it looks like that this is not directly possible?

We've used until now only 2 pipelines without a master pipeline, but we want to re-use the logic more. Currently we have 1 pipeline that calls the next pipeline and forwards the run_id.

Update 2023-03-31 As of early 2023 we can have output from a pipeline, via using the newly introduced system variable 'Pipeline Return Value'.

Official documentation is here: https://learn.microsoft.com/en-us/azure/data-factory/tutorial-pipeline-return-value

Upvotes: 16

Views: 15403

Answers (2)

Jason Welch
Jason Welch

Reputation: 986

ExecutePipline currently cannot pass anything from its insides to its output. You can only get the runID or name.

For some weird reason, the output of ExecutePipeline is returned not as a JSON object but as a string. So if you try to select a property of output like this @activity('ExecutePipelineActivityName').output.something then you get this error:

Property selection is not supported on values of type 'String'

I found that I had to use the following to get the run ID: @json(activity('ExecutePipelineActivityName').output).pipelineRunId

Upvotes: 10

Martin Esteban Zurita
Martin Esteban Zurita

Reputation: 3209

The execute pipeline activity is just another activity with outputs that can be captured by other activities. https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-pipeline-activity#type-properties

If you want to use the runId of the pipeline executed previosly, it would look like this:

@activity('ExecutePipelineActivityName').output.pipeline.runId

Hope this helped!

Upvotes: 2

Related Questions