How to build and test ASP.NET Core solution on Jenkins

Question

I want to build our ASP.NET Core solution on Jenkins continuous integration server.

The steps that I need run are:

1. dotnet restore
2. build solution: dotnet build or msbuild14 ?
3. run test: dotnet test
4. test coverage

Anybody knows or have the scripts to do the point 2 to 4?

Answer 1

You can use the following Pipeline code to run and publish the dotnet core test results:

node {
stage 'Checkout'
    cleanWs()
    checkout scm

stage 'Build'
    bat "\"C:/Program Files/dotnet/dotnet.exe\" restore \"${workspace}/YourProject.sln\""
    bat "\"C:/Program Files/dotnet/dotnet.exe\" build \"${workspace}/YourProject.sln\""

stage 'UnitTests'
    bat returnStatus: true, script: "\"C:/Program Files/dotnet/dotnet.exe\" test \"${workspace}/YourProject.sln\" --logger \"trx;LogFileName=unit_tests.xml\" --no-build"
    step([$class: 'MSTestPublisher', testResultsFile:"**/unit_tests.xml", failOnError: true, keepLongStdio: true])
}

I have uploaded some examples that I made to my GitHub for everyone to use and contribute, feel free to take a look:

https://github.com/avrum/JenkinsFileFor.NETCore

Those pipline jenkinsfile will add this pipline template to your build:

Answer 2

The first build action should be to execute dotnet restore. Then a PowerShell script is executed via a regular Windows Batch command:

powershell.exe -NoProfile -ExecutionPolicy Bypass ./TestsAndCoverage.ps1

I'm using the following script for unit testing and code coverage in Jenkins on Windows:

$testProjects = "Dangl.Calculator.Tests"
$testFrameworks = "net461", "net46", "netcoreapp1.0", "netcoreapp1.1"

# Get the most recent OpenCover NuGet package from the dotnet nuget packages
$nugetOpenCoverPackage = Join-Path -Path $env:USERPROFILE -ChildPath "\.nuget\packages\OpenCover"
$latestOpenCover = Join-Path -Path ((Get-ChildItem -Path $nugetOpenCoverPackage | Sort-Object Fullname -Descending)[0].FullName) -ChildPath "tools\OpenCover.Console.exe"
# Get the most recent OpenCoverToCoberturaConverter from the dotnet nuget packages
$nugetCoberturaConverterPackage = Join-Path -Path $env:USERPROFILE -ChildPath "\.nuget\packages\OpenCoverToCoberturaConverter"
$latestCoberturaConverter = Join-Path -Path (Get-ChildItem -Path $nugetCoberturaConverterPackage | Sort-Object Fullname -Descending)[0].FullName -ChildPath "tools\OpenCoverToCoberturaConverter.exe"

$testRuns = 1;

foreach ($testProject in $testProjects){
    foreach ($testFramework in $testFrameworks) {
        # Arguments for running dotnet
        $dotnetArguments = "test", "-f $testFramework", "`"`"$PSScriptRoot\test\$testProject`"`"", "-xml result_$testRuns.testresults"

        "Running tests with OpenCover"
        & $latestOpenCover `
            -register:user `
            -target:dotnet.exe `
            "-targetargs:$dotnetArguments" `
            -returntargetcode `
            -output:"$PSScriptRoot\OpenCover.coverageresults" `
            -mergeoutput `
            -excludebyattribute:System.CodeDom.Compiler.GeneratedCodeAttribute `
            "-filter:+[Dangl.Calculator*]* -[*.Tests]* -[*.Tests.*]*"

        $testRuns++
    }
}

"Converting coverage reports to Cobertura format"
& $latestCoberturaConverter `
    -input:"$PSScriptRoot\OpenCover.coverageresults" `
    -output:"$PSScriptRoot\Cobertura.coverageresults" `
    "-sources:$PSScriptRoot"

Short summary: The $testProjects variable defines (in this case one) test projects to run, they're assumed to be in the ./test folder relative to the script. It generates test results for all specified $testFrameworks (i.e. if I'm testing a netstandard1.3 library, I let it run against both netcoreapp and net).

In Jenkins, I chose Publish Cobertura Coverage Report with **/*Cobertura.coverageresults and Publish xUnit .Net Test Results with **/*.testresults. You need both the xUnit Plugin and the Cobertura Plugin.

The script requires that your test project has a (build time) dependency to both OpenCover and OpenCoverToCoberturaConverter NuGet packages, to generate the test results and transform them to the Cobertura format.

This is for Visual Studio 2015s project.json format

When you've already switched to Visual Studio 2017, you need to alter the test command: Run dotnet xunit instead of dotnet test. Include the dotnet xunit runner like this in your csproj file:

<PackageReference Include="xunit" Version="2.3.0-beta2-build3683" />
<DotNetCliToolReference Include="dotnet-xunit" Version="2.3.0-beta2-build3683" />

Check also that your build is configured to output debug symbols:

<PropertyGroup Condition="'$(Configuration)'=='Debug'">
  <DebugType>full</DebugType>
  <DebugSymbols>True</DebugSymbols>
</PropertyGroup>

The project is on GitHub if you want to take a further look.

This is only working on Windows so far. On Linux, everything except the code coverage is working.

Answer 3

The thing you have to keep in mind is which directory is jenkins executing from. dotnet restore can be run at the root, but dotnet build and dotnet test need to be run from the same directory as the project.json.

Test coverage is a separate topic altogether - as of right now (2/1/2017) in Visual Studio Enterprise 2015, code coverage does not work, at least with XUnit, maybe it does with MSTest. dotCover is working now, but I don't know how you could script that and get the results back.