CODEWITHSUNDEEP
testingazure-devopsautomated-testsazure-pipelinespull-request
Paul
Paul

Reputation: 259

DevOps Best Practice - how to run Pull Request Tests in parallel

We currently run PR tests in isolation i.e. when someone creates a PR, the PR tests are run against their changes. Once the tests are successful, the code is merged into main branch.

This allows us to test code in isolation, but it means we have to queue PR's up one after the other until the tests complete.

We could add another PR agent and allow PR's to run in parallel, but that could lead to code not being tested together. i.e. if 2 or 3 PR tests are running at once, it would only be testing that PR's code changes and not code that could be merged in by the other PR's.

So, my question is what is best practice in this situation and how can we go about running tests in parallel to lessen the queue time waiting for testing?

My question explained in a little more detail:

I think my question relates more to testing microservices because our UIs, apis, grpc etc all have their own repos and are deployed separately as microservices.

Our PR tests run tests against things like logging in and being able to perform certain functions which tests all these microservices are working.

If I have parallel PR tests running:

But, microservice1 changes alongside microservuce2 changes causes tests to fail. So code changes combined aren't tested in PR stage and can cause failing code to be released.

Upvotes: 1

Views: 661

Answers (2)

Rui Jarimba
Rui Jarimba

Reputation: 18094

how can we go about running tests in parallel to lessen the queue time waiting for testing?

The simplest option would be to slice the test suite so that each slice can be run independently, if possible.

For example, instead of running a large suite of 1000 tests on a single pipeline job, you can use 4 jobs and run 250 tests in parallel on each job.

Upvotes: 0

osim_ans
osim_ans

Reputation: 457

To follow best practices, you need to update the branch policies of the target(main) branch and add the build validation check set to your PR build. Under the same settings, you also need to set the build expiration to "Immediately when main is updated".

enter image description here

When a PR build runs, it runs against the 'merge branch' (a merged version of your private and main branch). This above setting makes the PR build policy status fail whenever the main branch is updated, and requeues a new build and after that only it can allow the PR merge. This ensures that the PR-changes build successfully even if the main branch changes, ensuring that the build always runs with the latest changes(or tests) from the main branch.

Thus, in your case, you can safely run multiple PR builds in parallel as they will always contain the latest changes. For example, if you create two PRs, the last successful PR build from PR1 will contain the tests from the main branch and the PR1 branch. Let's say PR1 is merged. PR2 will now contain tests from the main branch (which now includes the changes from PR1) and the PR2 branch.

You may refer this MS doc for details: https://learn.microsoft.com/en-us/azure/devops/repos/git/branch-policies?view=azure-devops&tabs=browser#build-validation

Upvotes: 0

Related Questions