CODEWITHSUNDEEP
azureazure-data-factory
Steven
Steven

Reputation: 639

Data Factory: how to stop pipeline execution without logic and without failure?

I have a simple pipeline consisting of:

When executed successfully, the notebook either outputs:

  1. An array of values for the ForEach loop (no issue here).
  2. A message that some conditions are not met (troublesome part).

In situation two a string is passed to the ForEach loop, which causes the activity to fail. This is not the desired result, because the pipeline should run successfully.

I have tried to solve the issue with the Switch activity, but you can not put a ForEach activity in a Switch. The same issue arises when I try to use the If Condition activity.

Any solutions and workarounds are welcome.

Upvotes: 0

Views: 727

Answers (1)

Aswin
Aswin

Reputation: 7116

As per Microsoft Document, For-each activity cannot be nested inside if activity or switch activity in Azure Data Factory (ADF). Instead, use the Execute Pipeline activity to create nested pipelines, where the parent has If activity and the child pipeline has for-each activity.

Limitation Workaround
You can't nest a ForEach loop inside another ForEach loop (or an Until loop). Design a two-level pipeline where the outer pipeline with the outer ForEach loop iterates over an inner pipeline with the nested loop.

Refer the NiharikaMoola-MT's answer on this SO thread. In the parent pipeline, keep Notebook activity and If activity. Then invoke the child pipeline with for-each activity inside the true section of If activity.

Upvotes: 2

Related Questions