CODEWITHSUNDEEP
c#automated-testsazure-devopsreportingdata-driven-tests
Stefan Black
Stefan Black

Reputation: 1

Efficient way to Report automated data-driven tests (csv) on vs-team-services

I have been searching for about 2 weeks but sadly couldn't find any efficient solution.

The UseCase isnt very tricky. I will try to discribe shortly:

The Situation:

I implemented automated C# Web-Service tests using Microsoft.VisualStudio.TestTools.UnitTesting . These tests are running nightly via a Build-Job on vs-team-services . Reporting is nice and traceable (epic/feature/task --> testcase --> bug) as long as i don't use data-driven tests. (I used test-suites and testcase-workitems and configured "associated automation" of eacht testcase)

The Problem is: If i use data-driven tests (csv-files) i can...

1) ...annotate the testmethods in the code with the Testcategory attribute and configure a test-assembly in the build, which will then run all testmethods with the assigned / configured Testcategory. In this case the testrun or build contains a report for every single data-row, which is fine. But the traceability is totally lost, because there are no testcases (vsts-workitems) that could be related to a task or bug.

2) ...create a testcase (workitem), add it to a test-suite/-plan and configure the build to run the suite / plan. Now the traceability is "fine", because the testcase can be related to a task or bug. But the report of the build contains only one single testcase, even the csv-file has e.g. 200 data-rows. Where each data-row contains data for one single test. So e.g, if one data-row (test) fails, the testcase (vsts-workitem) has failed, even if 199 tests succeeded.

What i am looking for:

The first approach with the possibility for traceability or the second approach with a reporting for each data-row.

I am wondering that this topic hasn't been discussed before and looking forward to new opportunities / approaches. Thanks in advance.

Upvotes: 0

Views: 173

Answers (1)

MrHinsh - Martin Hinshelwood
MrHinsh - Martin Hinshelwood

Reputation: 23444

You can add traceability from an automated test result directly to a Backlog Item. You can then see the requirements associated in the build, and create a Requirements Quality dashboard widget...

If you open a Build and go to the test results you can select an individual result, and then add a related work item. In all future builds the data will be related.

Then go to your dashboard and add the Requirement la Quality widget and configure it for that build.

Upvotes: 0

Related Questions