Reputation: 493
Azure - Run test project on agent from command line
I am working on Azure pipelines, running on a Windows self-hosted agent, behind a firewall to run automated tests.
I wrote questions on related issues last December in Dotnet test questions on errors and What .Net Core version to use on agent linked to the error message I had: ‘{"code":"UNABLE_TO_GET_ISSUER_CERT_LOCALLY"}’
I still have this error and I believe it could be linked to the firewall of the Agent but I tried to view the issue from a different angle.
I want to run a test project on a Dotnet Core 3.1 project, on a self-hosted agent, I have Dotnet SDK 5.0 and I saw a message about 3.1 being close to not supported anymore but I can’t migrate the application and I have others under 3.1 where I’ll need to build test pipelines.
I installed .Net Core 3.1 on the agent but being a .Net 2 developer, this is new to me, I found I have these versions on the agent:
C:\>dotnet --info
.NET SDK (reflecting any global.json):
Version: 5.0.100
Commit: 5044b93829
Runtime Environment:
OS Name: Windows
OS Version: 10.0.17763
OS Platform: Windows
RID: win10-x64
Base Path: C:\...\sdk\5.0.100\
Host (useful for support):
Version: 5.0.0
Commit: cf258a14b7
.NET SDKs installed:
3.1.301 [C:\...\sdk]
5.0.100 [C:\...\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.23 [C:\...\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.23 [C:\...\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.5 [C:\...\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.9 [C:\...\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.0 [C:\...\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.23 [C:\...\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.5 [C:\...\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.9 [C:\...\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.0 [C:\...\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.5 [C:\...\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.9 [C:\...\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.0 [C:\...\Microsoft.WindowsDesktop.App]
I thought that if the task in the pipeline is not able to run my test project, I should try from the command line on the agent server but then, I don’t know if I should set which version to use?
I tried this command:
E:\test>dotnet test E:\Agent\_work\01\s\src\SolutionFolder\ProjectFolder\TestProject.csproj
To get this output:
C:\dotnet\sdk\5.0.100\NuGet.targets(131,5): error : Unable to load the service index for source https://ourFeed.pkgs.visualstudio.com/33706eec-...4c4/_packaging/UpstreamFeed/nuget/v3/index.json.
C:\dotnet\sdk\5.0.100\NuGet.targets(131,5): error : Response status code does not indicate success: 401 (Unauthorized).
To connect to Nuget, we use a feed where nuget is configured so the pipelines are supposed to use this feed to connect to it, firewall blocks linking to Nuget directly. This works for build pipelines.
At the root of the solution, I have a nuget.config file, which I also copied in the test project as a test:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="ArtifactRepo" value=" https://ourFeed.pkgs.visualstudio.com/33706eec-...4c4/_packaging/UpstreamFeed/nuget/v3/index.json " />
</packageSources>
<config>
<add key="http_proxy" value="http://proxy.xxx:443" />
<add key="https_proxy" value="https://proxy.xxx:443" />
</config>
</configuration>
Am I right? I should first be able to run it from command line in order to have the pipeline working? Can I set it so it uses Dotnet Core 3.1 ?
When running the pipeline, I also get this: #[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
This is why I wanted to use Dotnet Core 3 on the agent, I am not sure what Nuget version I am using and if I can change that easily.
Can it be that, when running it locally from the command line, I am not using a PAT Token to connect to the feed?
If you had any idea to help me on this, thanks a lot. Claude
----------- UPDATE 1 -----------
I have checked that system environment variables exists and well set: HTTP_PROXY, HTTPS_PROXY and also: NO_PROXY for internal calls
On the agent server, I used your script but changed the version to the one that is installed on my developer machine:
dotnet new globaljson --sdk-version 3.1.301 --force
I ran the command: “dotnet test [path to csproj file]” and got an error I didn’t wrote down, it was either the ‘401 (Unauthorized)’ or ‘UNABLE_TO_GET_ISSUER_CERT_LOCALLY’
I added the nuget.config file I added above.
I tested the URL to the feed in a browser and was blocked by security, I added the URL to trusted sites, tested again the feed URL in a browser with same error.
I tested again “dotnet test [path to csproj file]” and now it worked !!
I removed the nuget.config and global.json files and it still worked !!!!
I don’t understand this, would it be the trusted sites or the fact that once, it read from the files and saved the required data somewhere… Maybe you’ll understand this.
The tests themselves are failing but the dotnet test command line works:
Failed: 29, Passed: 0, Skipped: 0, Total: 29, Duration: 1 s - MyProject.Test.Core.dll (netcoreapp3.1)
Now, running the pipeline, after adding a step with your powershell script, the global.json file gets created but I still have issues:
Results File: E:\Agent\_work\_temp\[***]_2021-02-05_11_53_38.trx
Failed! - Failed: 29, Passed: 0, Skipped: 0, Total: 29, Duration: 1 s - [***].Core.dll (netcoreapp3.1)
The 29 errors in the project are probably linked to security but I should get the error report in Azure right ?
I can see the TRX files generated in E:\Agent_work_temp\ but then they disappear and more errors appear, can it be security ? They told me they opened the way between that server and Azure… Would you have any ideas why I don’t get the tests report and code coverage report ?
There is still the warning about NET 5 having some compatibility issues. Here are the logs from where it starts to fail:
##[debug]Exit code 1 received from tool 'C:\Program Files\dotnet\dotnet.exe'
##[debug]Processed: ##vso[task.issue type=error;]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
##[debug]BuildConfiguration=undefined
##[debug]BuildPlatform=undefined
##[debug]testRunTitle=ProjectTests
##[debug]defaultRoot: 'E:\Agent\_work\_temp'
##[debug]findOptions.allowBrokenSymbolicLinks: 'false'
##[debug]findOptions.followSpecifiedSymbolicLink: 'true'
##[debug]findOptions.followSymbolicLinks: 'true'
##[debug]matchOptions.debug: 'false'
##[debug]matchOptions.nobrace: 'true'
##[debug]matchOptions.noglobstar: 'false'
##[debug]matchOptions.dot: 'true'
##[debug]matchOptions.noext: 'false'
##[debug]matchOptions.nocase: 'true'
##[debug]matchOptions.nonull: 'false'
##[debug]matchOptions.matchBase: 'false'
##[debug]matchOptions.nocomment: 'false'
##[debug]matchOptions.nonegate: 'false'
##[debug]matchOptions.flipNegate: 'false'
##[debug]pattern: '**/*.trx'
##[debug]findPath: 'E:\Agent\_work\_temp'
##[debug]statOnly: 'false'
##[debug]findPath: 'E:\Agent\_work\_temp'
##[debug]findOptions.allowBrokenSymbolicLinks: 'false'
##[debug]findOptions.followSpecifiedSymbolicLink: 'true'
##[debug]findOptions.followSymbolicLinks: 'true'
##[debug] E:\Agent\_work\_temp (directory)
##[debug] E:\Agent\_work\_temp\.taskkey (file)
##[debug] E:\Agent\_work\_temp\1a672b68-2897-472e-8574-05442c3ded4d.ps1 (file)
##[debug] E:\Agent\_work\_temp\[***]_2021-02-05_11_53_38.trx (file)
##[debug] E:\Agent\_work\_temp\Deploy_[***] 2021-02-05 11_53_37 (directory)
##[debug] E:\Agent\_work\_temp\Deploy_[***] 2021-02-05 11_53_37\In (directory)
##[debug] E:\Agent\_work\_temp\Deploy_[***] 2021-02-05 11_53_37\In\[***] (directory)
##[debug]7 results
##[debug]found 7 paths
##[debug]applying include pattern
##[debug]adjustedPattern: 'E:\Agent\_work\_temp\**/*.trx'
##[debug]1 matches
##[debug]1 final results
##[debug]Failed to get FF TestManagement.Server.UsePublishTestResultsLibInAgent Value.
Unable to get the FF: TestManagement.Server.EnablePublishToTcmServiceDirectlyFromTask. Reason: One or more errors occurred. (Moved Temporarily)
##[debug]Failed to get FF TestManagement.PTR.CalculateTestRunSummary Value.
##[debug]Reading test results from file 'E:\Agent\_work\_temp\[***]_2021-02-05_11_53_38.trx'
##[debug]Setting run start and finish times.
##[debug]Attachment location: E:\Agent\_work\_temp\[***]_2021-02-05_11_53_38\In
##[debug]Total test results: 29
##[debug]Processed: ##vso[results.publish type=VSTest;mergeResults=false;runTitle=[***];publishRunAttachments=true;resultFiles=E:\Agent\_work\_temp\[***]_2021-02-05_11_53_38.trx;testRunSystem=VSTS - dotnet;]
##[warning].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
##[debug]Processed: ##vso[task.issue type=warning;].NET 5 has some compatibility issues with older Nuget versions(<=5.7), so if you are using an older Nuget version(and not dotnet cli) to restore, then the dotnet cli commands (e.g. dotnet build) which rely on such restored packages might fail. To mitigate such error, you can either: (1) - Use dotnet cli to restore, (2) - Use Nuget version 5.8 to restore, (3) - Use global.json using an older sdk version(<=3) to build
Info: Azure Pipelines hosted agents have been updated and now contain .Net 5.x SDK/Runtime along with the older .Net Core version which are currently lts. Unless you have locked down a SDK version for your project(s), 5.x SDK might be picked up which might have breaking behavior as compared to previous versions. You can learn more about the breaking changes here: https://learn.microsoft.com/en-us/dotnet/core/tools/ and https://learn.microsoft.com/en-us/dotnet/core/compatibility/ . To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
##[debug]task result: Failed
##[error]Dotnet command failed with non-zero exit code on the following projects : E:\Agent\_work\31\s\src\[***]Core\[***].Core.csproj
##[debug]Processed: ##vso[task.issue type=error;]Dotnet command failed with non-zero exit code on the following projects : E:\Agent\_work\31\s\src\[***].Core\[***].Core.csproj
##[debug]Processed: ##vso[task.complete result=Failed;]Dotnet command failed with non-zero exit code on the following projects : E:\Agent\_work\31\s\src\[***].Core\[***].Core.csproj
##[section]Async Command Start: Publish test results
##[warning]Failed to publish test results: Moved Temporarily
##[section]Async Command End: Publish test results
##[section]Finishing: Tests
Many thanks again for your help, Claude
Upvotes: 0
Views: 904
Answers (1)
Reputation: 30313
I checked your other thread that Use dotnet task failed to install dotnet core 3.1 due to the agent is behind a proxy. As below is described in this document.
Note that many tools do not automatically use the agent configured proxy settings. For example, tools such as curl and dotnet may require proxy environment variables such as http_proxy to also be set on the machine
Please try adding two new environment variables called HTTP_PROXY and HTTPS_PROXY in your self-hosted agent machine. And set the value of those variables pointing to your proxy.
If above workaround didnot work out. Please try below workaround to set your agent to use dotnet core 3.1 specifically
As it describe in this document. The CLI searches for the global.json file in project directory . And use the dotnet version defined in this file.
You can run below dotnet command in your pipeline to create the "global.json" file in the current directory folder and set the dotnet version to use. See below example:
steps:
- powershell: |
dotnet new globaljson --sdk-version 3.1.9 --force
displayName: 'PowerShell Script'
Then you can run the test from both command line and dotnet task. You will see the dotnet 3.1 will be used to run your tests.
Upvotes: 1
Related Questions
- Azure DevOps Pipeline with selfhosted agent fails to run tasks
- Azure Devops MS Hosted Agent Pool Agent machine
- How to run Azure CLI tasks from an Azure DevOps Pipeline on a Self-Hosted Windows Agent?
- Azure Devops self-hosted agent with docker
- Azure Self-hosted Windows agent hook
- Installing Self-Hosted Agent remotely and wants to make it active agent and need to associate that to build pipeline
- Azure DevOps agent running an application with a dialog box
- Run MSBuild from Command Line task in Azure Devops
- Azure Devops - PowerShell task on self-hosted agent
- How do I run an arbitrary test command on a Hosted agent?