Execute Pipeline Not Passing Parameters From Parent To Child

Question

The purpose of the pipelines I'll outline below is to retrieve a list of inbound and outbound folder locations, and then move files from inbound to outbound based on a file naming pattern. We do this in SSIS with no issues.

I have a parent pipeline that retrieves a list of records from Snowflake, then passes that list to a ForEach loop. Inside the ForEach loop, I have an Execute Pipeline activity; the child pipeline called here is responsible for filtering the records and then using its own ForEach loop to go through the inbound location for the current record, grab the files matching the naming pattern, and then move on to the next record. Simple enough, right?

The parent pipeline has a lookup activity and a foreach loop. The ForEach loop has the Items property set as @activity('lookup').output.value. The Execute Pipeline activity inside the ForEach loop has 3 parameters defined and set thusly:

folderPath:  @item().FILE_PICKUP_LOCATION
destinationPath:  @item().FILE_DROPOFF_LOCATION
FILE_NAME_KEYWORD:  @item().FILE_NAME_KEYWORD

The child pipeline has the same 3 named parameters, all spelled the same, and all as string. Thank you, Microsoft, for not allowing nested ForEach loops in ADF.

Anyhow, when I debug/execute the parent pipeline, I can see that the input for the Execute Pipeline activity matches what is expected - values populated based upon the results of the query in the lookup activity. However, a check of the output shows that pipelineRunId and pipelineName are being sent, and not the parameters/values I've defined.

Anybody know why this is? I cannot find an answer in Microsoft's documentation, and Google is of no help. I've tried assigning the parameters without the open/close parens, but there is no change in the output. Why would the Execute Pipeline activity send the default system values but not the parameters? Is there some flag I forgot to set, or some line of json I can write to force this to happen?

Answer 1

Why would the Execute Pipeline activity send the default system values but not the parameters?

Execute pipeline activity or Dataflow activity won't give the input parameters in the activity's output. The values which are shown in the output were the executed child pipeline name and the child pipeline run id.

If you click on the pipeline run id, it will take to the child pipeline run where you can see the child pipeline activity run details.

Child pipeline run details:

If a child pipeline fails, you can go to this pipeline run and verify the activity runs and its outputs.

In the execute pipeline activity input, it will show the activity inputs which were given to it and not the child pipeline information.

The Execute pipeline activity is a like a function in a programming language. It will call the other pipeline, and it will pass values. It can also return the values from the child pipeline. You can check this documentation to know more about it.

And, coming to this error, this will occur when we give wrong dynamic content or special characters in the dynamic content expression.

In this sample. I have given an expression like below which caused the above error.

@json('{"name":"Rakesh"}').name#

To resolve this, re-check all your dynamic expressions and make sure you gave the correct dynamic expression without using the special characters like above.

If the issue still persists, then try to debug every activity only till that activity. This option will stop the debug after the activity execution. By using it for every activity, you can find which activity has the incorrect expression.