CODEWITHSUNDEEP
azure-devopsazure-data-factory
dave111
dave111

Reputation: 305

ADF Pipeline trigger deployment on DevOps

I'm doing some initial ADF deployment from my adf-dev to adf-staging environment. In the MS docs it says:

Deployment can fail if you try to update active triggers. To update active triggers, you need to manually stop them and then restart them after the deployment.

Does this mean I need to turn off my dev or staging triggers pre/post deployment?

2nd issue. I need to schedule the same set of triggers to run on different days in dev (sat) vs staging (sun). Do I need to make a separate set of triggers for each environment then or can I rewrite the trigger schedules for the existing triggers during deployment?

Upvotes: 2

Views: 4670

Answers (1)

Mutaz-MSFT
Mutaz-MSFT

Reputation: 806

You will need your staging triggers stopped before you start the deployment, and restarted after deployment is complete.

this page have a PowerShell script for stopping triggers: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#updating-active-triggers

Also you could use the custom petameters configuration file to update your trigger settings: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#triggers

To parametrise the trigger deployment in Arm template, first here is a sample weekly trigger that runs on a specific day:

{
"name": "OnceAWeekTrigger",
"properties": {
    "annotations": [],
    "runtimeState": "Stopped",
    "pipelines": [],
    "type": "ScheduleTrigger",
    "typeProperties": {
        "recurrence": {
            "frequency": "Week",
            "interval": 1,
            "startTime": "2021-05-25T22:59:00Z",
            "timeZone": "UTC",
            "schedule": {
                "weekDays": [
                    "Sunday"
                ]
            }
        }
    }
}
}

Create an arm-template-parameters-definition.json file as follow:

{

    "Microsoft.DataFactory/factories/triggers": {
        "properties": {
            "typeProperties": {
                "recurrence": {
                    "schedule": {
                        "weekDays": "=:-weekDays:array"
                    }
                }
            }
        }
    }
}

this file specifies that you want to prarametrise schedule_weekDays property.

after running ADFUtilities export function:

npm run build export c:\git\adf /subscriptions/<subscriptionid>/resourceGroups/datafactorydev/providers/Microsoft.DataFactory/factories/<datafactory_name> "ArmTemplate"

You now get arm template for trigger properties parametrised as follows:

  ...  {
        "name": "[concat(parameters('factoryName'), '/OnceAWeekTrigger')]",
        "type": "Microsoft.DataFactory/factories/triggers",
        "apiVersion": "2018-06-01",
        "properties": {
            "annotations": [],
            "runtimeState": "Stopped",
            "pipelines": [],
            "type": "ScheduleTrigger",
            "typeProperties": {
                "recurrence": {
                    "frequency": "Week",
                    "interval": 1,
                    "startTime": "2021-05-25T22:59:00Z",
                    "timeZone": "UTC",
                    "schedule": {
                        "weekDays": "[parameters('OnceAWeekTrigger_weekDays')]"
                    }
                }
            }
        }, ...

and the parameters file ArmTemplate\ARMTemplateParametersForFactory.json looks as follows:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "factoryName": {
            "value": "factory_name"
        },
        "OnceAWeekTrigger_weekDays": {
            "value": [
                "Sunday"
            ]
        }
    }
}

you could then create different parameter files for dev and staging with different days of the week by modifying the array value for OnceAWeekTrigger_weekDays

Upvotes: 2

Related Questions