CODEWITHSUNDEEP
selenium-webdriverazure-devopsazure-pipelinesspecflowui-testing
gagneet
gagneet

Reputation: 37259

SpecFlow BDD UI tests running on Azure DevOps pipeline

I have created a test suite using SpecFlow BDD with Selenium that I am trying to run on the Azure DevOps using Pipelines:

trigger:
  - master
pool:
  name: Hosted Windows 2019 with VS2019
  vmImage: windows-latest
schedules:
  - cron: '00 16 * * MON,TUE,WED,THU,FRI'
    displayName: Weekdays 4PM build
    branches:
      include:
        - master
        - releases/*
      exclude:
        - development
        - feature/*
jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: windows-latest
    steps:
      - task: PowerShell@2
        displayName: Clone the pipelines repository
        inputs:
          targetType: inline
          script: >
            git clone -b master
            https://azuredevops:$($env:token)@dev.azure.com/abc-comapny/MyAzure/_git/MyProject.Builder
            pipeline-repo/
        env:
          token: $(System.AccessToken)
      - task: Bash@3
        displayName: Prepare the nuget.config file
        inputs:
          targetType: inline
          script: |
            echo copying the nuget.config to the root folder
            cp ./pipeline-repo/templates/nuget.config .
      - task: NuGetCommand@2
        displayName: Authenticate with Azure DevOps NuGet
        inputs:
          command: custom
          arguments: >-
            sources update -Name "Azure DevOps" -Username "vsts" -Password
            "$(System.AccessToken)" -StorePasswordInClearText -ConfigFile
            ./nuget.config
      - task: UseDotNet@2
        displayName: Using SDK Version (3.1.x)
        inputs:
          version: 3.1.x
      - task: DotNetCoreCLI@2
        displayName: Restore
        inputs:
          command: restore
          projects: '**/*.sln'
      - task: DotNetCoreCLI@2
        displayName: Build the solution
        inputs:
          command: build
          projects: '**/*.sln'
      - task: SpecFlowPlus@0
        inputs:
          projectFilePath: .
          projectName: MyProject.Qa
          projectLanguage: en
          workItemPrefix: AUTO
      - task: DotNetCoreCLI@2
        displayName: Publish
        inputs:
          command: publish
          publishWebProjects: false
          projects: '**/*.sln'
          arguments: >-
            --configuration Release -o
            $(build.artifactstagingdirectory)/SeleniumTests
          zipAfterPublish: false
          modifyOutputPath: false
      - task: PublishBuildArtifacts@1
        displayName: Publish Artifact
        inputs:
          PathtoPublish: $(build.artifactstagingdirectory)
        condition: succeededOrFailed()
      - script: >-
          dotnet test MyProject.Qa.sln -r "$(Agent.TempDirectory)/" --logger
          "trx;LogFilename=testresults.trx" 
        displayName: Run Unit Tests and Code Coverage
        continueOnError: false

I have tried looking up the instructions given at:

https://swimburger.net/blog/dotnet/how-to-run-net-core-selenium-ui-tests-on-azure-devops-pipelines

https://www.stuartwhiteford.com/running-selenium-ui-tests-in-an-azure-devops-pipeline/

https://azuredevopslabs.com/labs/vstsextend/selenium/#:~:text=Navigate%20to%20Pipelines%20under%20Pipelines,successful%2C%20release%20will%20be%20triggered.

The steps are all passing, but I do not get any result or see any output for the test results. I have 2 projects, both MS Test, one for the BDD Specifications and the other for the Selenium framework files:

Has anyone used SpecFlow BDD UI tests with Azure DevOps Pipeline to run the UI tests in a non-headless browser type run? If so, could you inform, what you did and if I am doing anything wrong with the above 'azure-pipeline.yaml' file and need to change anything to run these tests?

Also, how do I use the above to send across a mail to all the stakeholders? Can I do that with the YAML or do I need to do that from some other part of the Azure DevOps, once I have the report up?

Upvotes: 1

Views: 3904

Answers (1)

Andreas Willich
Andreas Willich

Reputation: 5825

I think you are missing a task to publish the test results to Azure DevOps. You are executing dotnet test and get a trx- file (the result file), but you are not doing anything with it.

You need to use the Publish Test Result task (https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml) afterward to upload the trx- file to Azure DevOps.

Or you change the dotnet test command to use the Visual Studio Test task (https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/test/vstest?view=azure-devops). This tasks makes the upload of the trx- file automatically.

Upvotes: 2

Related Questions