CODEWITHSUNDEEP
azureazure-data-factory
Suraj Shejal
Suraj Shejal

Reputation: 650

ADF best practice for Dataflow in Parallel

I have one json file which has approx 200 lists which I'm passing to ForEach activity. Which calls for lookup then DataFlow .

Currently I'm using TTL 30min AZ Integration Runtime which took 30min to complete.

I'm want to know if i Run Sequentially will the cost differ or same as without Sequential ForEach.

What should be the best approach.

Upvotes: 0

Views: 795

Answers (1)

Mark Kromer MSFT
Mark Kromer MSFT

Reputation: 3838

You can save a bit if you run sequentially because you'll only incur the cluster cold startup of 4 mins on the initial run. The subsequent sequential runs will then start immediately and save you ~4 mins for each one.

Upvotes: 0

Related Questions