CODEWITHSUNDEEP
deploymentcontinuous-integrationazure-rm-templateazure-data-factory
Manish Sharma
Manish Sharma

Reputation: 21

Self Hosted IR unavailable after ARM deployment

We are trying to use self hosted integration runtime to extract data from on-prem fileshare. To implement CI/CD, I have created arm templates from the data factory where IR is successfully working and enabled sharing on for the Data Factory in which I am going to deploy my pipelines using ARM templates. I can successfully deploy pipeline and self hosted IR and linked services but IR is not available in the new data factory connections.

enter image description here

Is it normal? Because to use CI/CD with Data Factory, as soon as ARM gets deployed we should be ready to run pipelines without manual changes? And if I am correct then can anyone help why IR in the new Data Factory isn't available which is making the pipeline failed when I am trying to run it.

Upvotes: 1

Views: 1234

Answers (2)

MartinJaffer-MSFT
MartinJaffer-MSFT

Reputation: 728

A Self-Hosted Integration Runtime, is 'owned' by exactly one Data Factory instance. The way the 'owner' and the 'sharer' factories define the IR are different. When you deployed one over the other, the type changed and you ended up with either two 'owners' or two 'sharers'. Since there can only be one 'owner' or a 'sharer' points to an 'owner', things break.

Upvotes: 0

Anand Maurya
Anand Maurya

Reputation: 101

Self Hosted Integration Runtime are tied to the ADF it is created in. To use CI/CD with Self Hosted IR you need to do following steps:

  1. Create a new Data Factory other than the ones you using in CI/CD process,then create the Self hosted Integration Runtime their.(This ADF doesn't need to contain any of your pipeline or Dataset).
  2. Go to the newly created Integration Runtime and click on edit or pencil icon. Go to sharing tab of opened window.
  3. Click on Grant Permission to other Data factory.(Search and Give Permission to all ADF involved in CI/CD Process).
  4. Copy the resource id Displayed. Go to the DEV Data Factory and create new Self hosted runtime of type linked.

enter image description here

5.Enter the Resource ID when asked and click create.

6.Then proceed to setup CI/CD process through DEV Data Factory.

Since through ARM template in all other Data factory linked Self Hosted IR will be created and if you provided permission then everything will work.

Upvotes: 1

Related Questions