CODEWITHSUNDEEP
unit-testingcontinuous-integrationazure-devops
Chris Hammond
Chris Hammond

Reputation: 2186

VSTS / VSTest not adhering to [ExcludeFromCodeCoverage]

I have just recently started using Continuous Integration with VSTS and I have set up a pipeline which includes the "Visual Studio Test" Task.

Within the task, there's an option to execute a code coverage scan as part of the test.

To ensure that my code coverage only covers MY code and have a) created a .RunSettings file to include only the assemblies I generate and b) there are some parts of the code with the [ExcludeFromCodeCoverage] attribute.

Now, when execute Analyze Code Coverage from Visual Studio (2017 Enterprise, 15.7.4), everything works as I would expect, only my assembly gets analyzed and the code I have excluded is, errr, excluded.

However, when the VSTS pipeline is run, no such limitations are applied and ALL assemblies are tested and ALL code, including the specifically excluded code, which then results in a dramatic drop on the code-coverage %age.

Not sure where to go from here so hoping the S/O community can help.

Update - RunSettings file in use

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Configurations for data collectors -->
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <ModulePaths>
              <Include>
                <ModulePath>.*\.dll$</ModulePath>
              </Include>
              <Exclude>
                <ModulePath>.*Tests.dll$</ModulePath>
                <ModulePath>.*moq.dll$</ModulePath>
              </Exclude>
            </ModulePaths>
            <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
          </CodeCoverage>
        </Configuration>
      </DataCollector>

    </DataCollectors>
  </DataCollectionRunSettings>

Upvotes: 1

Views: 860

Answers (1)

Andy Li-MSFT
Andy Li-MSFT

Reputation: 30412

There is an issue here: RunSettings file not used in TFS VsTest task, you can go and check the discussion about this issue.

As a workaround you can use below .RunSettings formats (add the UseVerifiableInstrumentation = False) 

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
           <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
            <ModulePaths>
              <Include>
                <ModulePath>.*\\MyProjectName.*\.dll$</ModulePath>
              </Include>
              <Exclude>
                <ModulePath>.*Tests.dll$</ModulePath>
              </Exclude>
            </ModulePaths>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

Just check acesiddhu's explanation:

you are using visual studio test tools installer task in your definition. in case that task is used useverifiable property needs to be set to false because in xcopy mode we don't GAC the above dll 'Microsoft.VisualStudio.CodeCoverage.Shim (this is a dependency which needs to be loaded when useverifiable is set to true)

marking this property false ensures it doesn't use this particular dll.

Upvotes: 1

Related Questions