How to setup project level parameters in Azure Data Factory V2

Question

We need to setup project level parameters which will used repeatedly across each of the pipelines in ADF V2. Currently we are repeating the same parameters in each pipeline. If at all we need to modify the parameter, we have to go manually into each of the pipeline's trigger and amend them which is tedious and erroneous.

is there any better way to setup them as project level parameters and they'll automatically initialised and assigned across the Data factory whenever pipeline triggers.

Answer 1

Thinking on the fly here, but have you considered reading a DB to get this parameter?

You can make every pipeline to do a lookup (LookupActivity) on a table where there is only one record with this parameter, and then set it (Set Variable Activity) as a variable, then use this variable. If at all you have to change this parameter, just change the record in the database and all pipelines will immediatly use this new value in their next executions.

Hope this helped!

Answer 2

Based on the official document, A pipeline is a logical grouping of activities that together perform a task. No more thing represents the group of pipelines so that you can't let pipelines share one same parameter.

I have a workaround,using Execute pipeline activity and Set Variable activity.

You could invoke other pipelines in a Execute pipeline activity.Before that, you could set a variable in the pipeline.

Then reset the value in the Set Variable activity to provide the same parameter for all the pipelines.

Some details,please refer to this link.