Azure pipelines: abstract away parameters

Question

I have a pipeline that deploys the application to kubernetes, however I strongly dislike having to give the connection type, azure subscription endpoint, resource group, namespace, cluster, etc. for every single command. As such, I'm looking for a way to make this more generic. I tried the following as a template:

parameters:
- name: displayName
  type: string
- name: command
  type: string
- name: arguments
  type: string
  default: ''
- name: configuration
  type: string
  default: ''

steps:
- task: Kubernetes@1
  displayName: ${{parameters.displayName}}
  inputs:
    command: ${{parameters.command}}
    arguments: ${{parameters.arguments}}
    configuration: ${{parameters.configuration}}
    connectionType: ${{variables.connectionType}}
    azureSubscriptionEndpoint: ${{variables.azureSubscriptionEndpoint}}
    azureResourceGroup: ${{variables.azureResourceGroup}}
    kubernetesCluster: ${{variables.kubernetesCluster}}
    namespace: ${{variables.namespace}}
    containerRegistryType: ${{variables.containerRegistryType}}

However, it seems variables imported in the main pipeline (which references this template) are not propagated (as confirmed by https://stackoverflow.com/a/75458750/14182569). Furthermore, as this template only references a task, it seems syntactically illegal to import variable template files there. Since the templates are compiled, something like the replacetokens task also doesn't help. Adding all of these values as explicit parameters each time I refer to this task seems extremely inefficient, copying code is the first thing I was taught not to do. Is there a proper way to tackle this? Putting all steps as separate jobs might allow me to import the variable template files, but is that good practice?

Answer 1

I have a pipeline that deploys the application to kubernetes, however I strongly dislike having to give the connection type, azure subscription endpoint, resource group, namespace, cluster, etc. for every single command.

One way or another you'll have to use all these parameters and/or variables in some of your templates - the question is where.

Generally speaking, I think you should hide all the implementation details behind a job, which should be simple and have one and only one reponsibility - in your case, you can create a job template that includes all the steps required to deploy a kubernetes application.

Sample pipeline

Using the job template in a pipeline could be done like this:

# my-pipeline.yaml

parameters:
  # other parameters here (agent pool, job timeout, etc)

  - name: environment
    type: string
    displayName: 'Environment'
    default: test
    values:
      - test
      - qa
      - prod

  - name: dryRun
    type: boolean
    displayName: 'false to deploy changes; true to run in validation mode only'
    default: true

jobs:
  - template: /pipelines/jobs/kubernetes/deploy-application-job.yaml
    parameters:
      # other parameters here (agent pool, job timeout, etc)
      environment: ${{ parameters.environment }}
      dryRun: ${{ parameters.dryRun }}

  # other jobs here

As you can see, there aren't many parameters passed to the job template. Reducing the number of environment-related parameters makes the code more readable. Also, it's easier to reuse the job in multiple pipelines or contexts, such as:

• Deploying the application to a cluster
• Running it as part of a pull request validation (by setting dryRun to true).

---

Job template implementation

# /pipelines/jobs/kubernetes/deploy-application-job.yaml

parameters:
  # other parameters here (agent pool, timeout, etc)

  - name: environment
    type: string
    displayName: 'Environment'

  - name: dryRun
    type: boolean
    displayName: 'false to deploy changes; true to run in validation mode only'
    default: true

jobs:
  - job: kubernetes_${{ parameters.environment }}
    displayName: 'Deploy to Kubernetes'
    # other job settings here
    variables:
      # Consumers of this job are expected to provide a variables template 
      # using the following folder structure:
      # /pipelines/variables/kubernetes/{environment}-variables.yaml
      - template: /pipelines/variables/kubernetes/${{ parameters.environment }}-variables.yaml@self
    steps:
      - template: /pipelines/steps/kubernetes/deploy-application-steps.yaml
        parameters:
          authentication:
            azureSubscriptionEndpoint: ${{ variables.azureSubscriptionEndpoint }}
            azureResourceGroup: ${{ variables.azureResourceGroup }}
            # other parameters here

Notes:

• Environment-related parameters such as environment are used as part of the variables template referenced in the job:

    /pipelines/variables/kubernetes/${{ parameters.environment }}-variables.yaml@self
  

• Referencing the variables template at the job level reduces the variable scope, which allows the same job to be reused.

• Variables are referenced at the job level only (IMO it's better to avoid using variables in steps templates)

---

Steps template implementation

Steps template uses parameters only in order to remove dependencies to variables:

# /pipelines/steps/kubernetes/deploy-application-steps.yaml

parameters:
  - name: displayName
    type: string

  # other parameters here such as azureSubscriptionEndpoint, azureResourceGroup, etc

steps:
# All steps here required to setup and deploy the application

# ...

- task: Kubernetes@1
  displayName: ${{ parameters.displayName }}
  inputs:
    command: ${{ parameters.command }}
    arguments: ${{ parameters.arguments }}
    configuration: ${{ parameters.configuration }}
    connectionType: ${{ parameters.connectionType }}
    azureSubscriptionEndpoint: ${{ parameters.azureSubscriptionEndpoint }}
    azureResourceGroup: ${{ parameters.azureResourceGroup }}
    kubernetesCluster: ${{ parameters.kubernetesCluster }}
    namespace: ${{ parameters.namespace }}
    containerRegistryType: ${{ parameters.containerRegistryType }}

Note:

• Consider grouping related parameters if you have a long list of parameters.

---

Variables template

Finally, the variable templates could be organized by component and environment:

# /pipelines/variables/kubernetes/qa-variables.yaml

variables:
  - name: azureSubscriptionEndpoint
    value: myQaSubscription
  
  - name: azureResourceGroup
    value: myQaResourceGroup

  # other variables

# /pipelines/variables/kubernetes/prod-variables.yaml

variables:
  - name: azureSubscriptionEndpoint
    value: myProdSubscription
  
  - name: azureResourceGroup
    value: myProdResourceGroup

  # other variables