CODEWITHSUNDEEP
azure-data-factory
Priya Jha
Priya Jha

Reputation: 111

Parallel ForEach Iteration for Execute Pipeline activity

I want to execute a pipeline in parallel via ForEach Activity.

Below is the sample code for parent and child pipeline.

 {
"name": "Parent",
"properties": {
    "activities": [
        {
            "name": "ForEach1",
            "type": "ForEach",
            "dependsOn": [],
            "userProperties": [],
            "typeProperties": {
                "items": {
                    "value": "@pipeline().parameters.Test",
                    "type": "Expression"
                },
                "isSequential": false,
                "activities": [
                    {
                        "name": "Execute Pipeline1",
                        "type": "ExecutePipeline",
                        "dependsOn": [],
                        "userProperties": [],
                        "typeProperties": {
                            "pipeline": {
                                "referenceName": "Child",
                                "type": "PipelineReference"
                            },
                            "waitOnCompletion": true
                        }
                    }
                ]
            }
        }
    ],
    "concurrency": 10,
    "parameters": {
        "Test": {
            "type": "array",
            "defaultValue": [
                1,
                2
            ]
        }
    },
    "annotations": []
}

}

{
"name": "Child",
"properties": {
    "activities": [
        {
            "name": "Wait1",
            "type": "Wait",
            "dependsOn": [],
            "userProperties": [],
            "typeProperties": {
                "waitTimeInSeconds": 120
            }
        }
    ],
    "concurrency": 10,
    "annotations": []
}

}

After executing the pipeline, ForEach iteration is executing sequentially rather than parallelly therein executing child pipeline sequentially. Is there any configuration changes that i am missing out which is causing child pipeline to run sequentially.

Upvotes: 1

Views: 3585

Answers (1)

Nandan
Nandan

Reputation: 4945

My assumption is you are executing the pipeline in debug mode : enter image description here Because the same code I executed via trigger, it was as expected wherein the execute pipeline was in parallel

enter image description here

Note: The updated the parameter to 7 iterations hence you are able to see more child pipelines

Upvotes: 3

Related Questions