Merge a pull request if and only if a build succeeds

Question

When doing configuration-as-code and/or infrastructure-as-code, the problem often is that something being committed in the version control does not mean it is also applied on the environment. But there is a way to ensure it (except for very untimely network failures): push the intended changes in a branch and have the CD server apply the configuration and push it in master if and only if it applies.

So I have found a way to set up the DevOps repository so that a pull-request can only be merged if a build succeeded (1). And such build is useful for validating the syntax and previewing the changes (tf plan, kubectl diff and similar).

But then I still need to actually complete the pull request from the build.

Ensuring the PR can't be completed in any other way can probably be done with the policy or permissions, and ensuring the pull request is otherwise ready (reviewed) can be done by replacing the review approvals with approvals in the pipeline.

Answer 1

It does not handle the completely general case, but for running terraform there is Atlantis.

The way it works is that it watches the pull requests, and on a special comment (atlantis apply) it executes terraform apply and if it succeeds (and if it's configured that way) it merges the pull request. So then if only the technical user it runs under is given permission to commit&push to master, it does the right thing.

Approximately the same workflow can be used with builds. An identity can be given to that specific pipeline to merge the PR and do merges by triggering the build.

But the main motivating use-case for this is infrastructure-as-code, and infrastructure-as-code is mainly terraform anyway, so this should handle large part of the practical uses.

Answer 2

The solution I can think if is a combination of the following:

1. In your build pipeline, set the completion status of the pull request to auto complete, by calling the Azure DevOps REST API and setting the AutoCompleteSetBy.Id to whoever user you want. This will auto-enable the autocomplete on the pipeline.
2. In the branch policies for the merge to the target branch (in your case master), add a merge policy requirement that requires your build pipeline to be successful to allow a merge of your PR.

This will result in:

• Your pipeline will auto-complete once all branch policy checks are successful.
• Your build pipeline is required to be successful for the pipeline to complete.

Answer 3

Completing an Azure DevOps Pull Request "if and only if" a build succeeds is an interesting question. I believe it's possible.

The "only if" part is easy; simply turn on branch policies for the target branch (master in your case). It doesn't actually matter what policy you enable, as long as it's something a user can't achieve. For example, you could set the minimum number of reviewers to 10, or you could add a required reviewer which is an account no one can impersonate, or you can require a build or status policy, or you can use some combination of these. Basically you just need to make sure no one can complete a PR into this branch unless they have the bypass security setting. (So obviously don't give anyone that privilege either.)

Now for the "if" part; you could add a post build step which completes the PR with bypass using the API. (Here's a possible working example from the AzureDevOps dev forum.) Using the bypass should force complete the PR with two possible exceptions that I can think of:

1. A user could abandon the PR.
2. A user could vote to reject the PR. (Or select "Waiting for Author".)

Even these could likely be overcome via the API, but presumably if one of these occurs the user likely has a good reason and you probably don't want the PR to complete anyway.

Answer 4

You can always have your pipeline run the merge :

• instead of running your tests on the HEAD commit of the branch,
• have your job switch to your target branch, run git merge <commit>, and run your tests on that.