CODEWITHSUNDEEP
unit-testingazure-devopsmstest
willem
willem

Reputation: 27027

How can I configure MSTest to fail the unit test run on any warnings?

Our unit tests execute in a Visual Studio Online build using the "Visual Studio Test" task. Occasionally developers change paths, which cause no tests to be executed at all.

We get warnings in the Visual Studio Online logs, but the build still succeeds, so it's hard to spot.

2017-10-02T16:33:15.7470931Z ##[warning]No test assemblies found matching the pattern: **\DBX.Tests*.dll;-:**\obj\**.

Is there any way I can configure MSTest or the "Visual Studio Test" task in VSO to fail if there are warnings?

Upvotes: 1

Views: 431

Answers (1)

Marina Liu
Marina Liu

Reputation: 38096

You can use other ways to failed the build result when the visual studio test task has such warning. Detail steps as below:

1. Add a PowerShell task after Visual Studio Test task.

2. Then get Visual Studio Test task build information by Timeline:

GET https://account.visualstudio.com/DefaultCollection/{project}/_apis/build/builds/{buildID}/timeline?api-version=2.0

For the buildID, you can get it through predefined variable $(Build.BuildId).

3. Search for Visual Studio Test task information by task name or task display name.

Such as part of response of the rest api as below:

{
  "id": "59863cf3-dc46-4e1b-914a-6e903e1aa924",
  "parentId": "95930a52-e560-42d6-b9d2-14e0eef98758",
  "type": "Task",
  "name": "VsTest - testAssemblies",
  "startTime": "2017-10-03T15:54:36.35Z",
  "finishTime": "2017-10-03T15:54:37.897Z",
  "currentOperation": null,
  "percentComplete": null,
  "state": "completed",
  "result": "succeeded",
  "resultCode": null,
  "changeId": 16,
  "lastModified": "0001-01-01T00:00:00",
  "workerName": "Hosted Agent",
  "order": 5,
  "details": null,
  "errorCount": 0,
  "warningCount": 2,
  "url": null,
  "log": {
    "id": 6,
    "type": "Container",
    "url": "https://maaccount.visualstudio.com/DefaultCollection/f7855e29-6f8d-429d-8c9b-41fd4d7e70a4/_apis/build/builds/1583/logs/6"
  },
  "task": {
    "id": "ef087383-ee5e-42c7-9a53-ab56c98420f9",
    "name": "VSTest",
    "version": "2.1.12"
  },
  "issues": [
    {
      "type": "warning",
      "category": "General",
      "message": "",
      "data": {
        "type": "warning",
        "code": "002004"
      }
    },
    {
      "type": "warning",
      "category": "General",
      "message": "No test assemblies found matching the pattern: **\\*test*.dll,!**\\obj\\**.",
      "data": {
        "type": "warning"
      }
    }
  ]

}

You can search by the task name VSTest or the task display name VsTest - testAssemblies.

For the issues object, you can get the warnings you need.

4. Fail the build result if VS test has the warning you specified.

If there has the warning as you specified (such as the warning No test assemblies found matching the pattern: **\DBX.Tests*.dll;-:**\obj\**), you can fail the PowerShell task by 

exit 1

Upvotes: 1

Related Questions