CODEWITHSUNDEEP
azure-devopsterraformazure-yaml-pipelines
Nick
Nick

Reputation: 25798

Prevent Cancellation of Deployment Job on PR Changes

We have an Azure DevOps YAML multi-stage pipeline where code is built and then deployed to a sequence of environments. Deployment is achieved using Terraform.

i.e.

Builds -> Test -> Deploy to DEV -> Deploy to TEST - - ->

This pipeline is used for both CI/CD builds and also PR builds. For the PR build, the only part of the deployment stage is running terraform plan on the TF scripts.

For PRs, the pipeline is configured to cancel the ongoing build if changes are pushed to that PR.

The problem we're seeing is that when changes are pushed to the PR and the ongoing build is cancelled, sometimes that cancellation happens during the terraform plan step. This occasionally means that the blob lease taken by terraform plan is not released. From that point onwards, manual intervention is required (break the blob lease) in order for the deployment stages to run successfully.

I believe we can switch off the setting which causes the ongoing PR build to be cancelled if changes are pushed.

But I wondered if there was a way of marking a pipeline step as critical - i.e. it should run to completion if the build is cancelled?

There are other ways of cancelling a pipeline build and there must be other tasks/steps which should not be cancelled part-way through. Such a critical-task setting would cover these situations too.

Upvotes: 4

Views: 1012

Answers (2)

Tom
Tom

Reputation: 21

Not sure if you solved this, but I had the exact same issue. Adding condition: always() to my task forced it to complete when DevOps cancelled the pipeline after additional changes were pushed.

See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml:

jobs:
- job: Foo

  steps:
  - script: echo Hello!
    condition: always() # this step will always run, even if the pipeline is canceled

Upvotes: 2

Krzysztof Madej
Krzysztof Madej

Reputation: 40779

I'm afraid you won't be able to achieve this using just YAML. What you can do will require some additional effort (and in some cases quite big):

  • replace terraform script with bicep for isntance - for me syntax iq quite similar and what you get here is lack of terrafrom state
  • add in your very first step a check if your state is locked and break a lease if it is needed

I understood that you would like to hear something better than this but at the moment there is no way to mark a task as non-cancellable. However, this sound like a cool feature and candidate for feature request.

Upvotes: 0

Related Questions