CODEWITHSUNDEEP
githubazure-devopsazure-data-factorycontinuous-deploymentcicd
Pietro D'Antuono
Pietro D'Antuono

Reputation: 412

Retain GitHub repository after Azure Data Factory pipeline deployment

The project

I have an Azure Data Factory (ADF) pipeline that gets automatically deployed through an Azure DevOps CI/CD pipeline whenever some changes are detected.

Azure DevOps pipeline

The problem

Whenever the deployment pipeline runs, the GitHub gets disconnected from ADF

The topic has been asked already in these two stackoverflow questions (that I know of):

but I still cannot figure out where to add the repoConfiguration options (in the arm-template-parameters-definition.json? how?) so that when I publish from ADF, the ARMTemplateForFactory.json includes the repoConfiguration options.

What I tried

I tried editing the arm-template-parameters-definition.json so that it includes something like the following (as per this Microsoft Learn page):


{
  "Microsoft.DataFactory/factories": {
    "properties": {
      "globalParameters": {
        "*": {
          "value": "="
        }
      },
      "globalConfigurations": {
        "*": "="
      },
      "encryption": {
        "*": "=",
        "identity": {
          "*": "="
        }
      },
      "repoConfiguration": {
        "accountName": "=",
        "repositoryName": "=",
        "collaborationBranch": "=",
        "rootFolder": "=",
        "clientId": "=",
        "clientSecret": {
          "byoaSecretAkvUrl": "=",
          "byoaSecretName": "="
        },
        "hostName": "="
      }
    },
    "location": "=",
    "identity": {
      "type": "=",
      "userAssignedIdentities": "="
    }
  }
}

No success, though. Is this a good approach? What else should I do?

My last alternative

If nothing else works, my last alternative could be adding an InvokeRESTAPI@1 task in the Azure DevOps deployment pipeline to reconnect ADF to GitHub, since Factories_ConfigureFactoryRepo allows it. I do not like that much this alternative as it only bypasses the problem, but does not solve it.

# Create a new task that runs an HTTP request that connects ADF to the GitHub repo
# https://learn.microsoft.com/en-us/rest/api/datafactory/factories/configure-factory-repo?tabs=HTTP
- task: InvokeRESTAPI@1

  inputs:
    connectionType: 'AzureRM'
    azureSubscriptionEndpoint: 'MyAzureServiceConnection'
    method: 'POST'
    url: 'https://management.azure.com/subscriptions/************/providers/Microsoft.DataFactory/locations/West Europe/configureFactoryRepo?api-version=2018-06-01'
    body: |
      {
        "factoryResourceId": "/subscriptions/************/resourceGroups/resource_name/providers/Microsoft.DataFactory/factories/data-factory-name",
        "repoConfiguration": {
          "type": "FactoryGitHubConfiguration",
          "accountName": "GitHubAccountName",
          "collaborationBranch": "main",
          "repositoryName": "GitHubRepoName",
          "rootFolder": "/",
          "clientId": "client-id",
          "clientSecret": "client-secret",
        }
      }
    contentType: 'application/json'

Upvotes: 0

Views: 298

Answers (1)

Regina M
Regina M

Reputation: 1

I could not find much information on this; however, it looks like Microsoft only recommends you release to QA and Prod. Please review Current CI/CD flow here: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-improvements

I Did read a similar approach to your Solution #1 here: [https://stackoverflow.com/questions/56854139/azure-arm-template-deployment-of-datafactory-with-azure-devops-git-configuration/56863897#56863897][1]

Upvotes: 0

Related Questions