CODEWITHSUNDEEP
azureazure-data-factory
Manas R
Manas R

Reputation: 13

How do i implement a pipeline where the activity inside for each runs for a specific subset of the look up value output array?

I have an array output (len = 503 ) from the lookup value activity which is basically fetching strings from a CSV file. These strings are parameters for my dataflow activity within the for each.

Current adf pipeline:

current adf pipeline as shown

I now want to run a dataflow within for each of the first n strings in the array. I would like to run the pipeline for the first 50 or the last 100. How do I subset the array of lookup activity and run iterate over the dataflow inside the for each dynamically or change over time? Snippets would be appreciated.

I tried using range func within the items tab on for each activity and used @item.'name' to fetch the current value in that iteration

Upvotes: 0

Views: 143

Answers (2)

Rakesh Govindula
Rakesh Govindula

Reputation: 11464

As you want to give an array of first n elements from lookup array, you can use take() in the ForEach expression.

Here, my lookup output array length is 10 and I am giving first 5 elements to the ForEach.

@take(activity('Lookup1').output.value,5)

enter image description here

and you can see it took the first 5 elements from the lookup array by looking at number of iterations.

enter image description here

If you want to take last n strings array, use skip().

Here, I want to give last 6 elements from the lookup array. That means my starting value should be from 5th element i,e index 4. So, give the index of the starting value and it will give desired array by skiping the elements before the index.

@skip(activity('Lookup1').output.value, 4)

enter image description here

Upvotes: 1

Christian
Christian

Reputation: 5531

Examples using range:

@map(range(0, 50), function(index) { return activity('YourLookupActivityName').output.value[index]; })

@map(range(length(activity('YourLookupActivityName').output.value) - 100, 100), function(index) { return activity('YourLookupActivityName').output.value[index]; })

Upvotes: 0

Related Questions