Run Azure DevOps test case with REST APIs

Question

I have a requirement to run automated API test (associated to a test case) in Azure Devops.

I can run it manually by selecting - build and release.

Since I'm running the testcases from Azure Devops yaml pipeline on a Linux agent, I cannot use VSTEST@2 task to input - TestPlan , TestSuite id's.

The only option I found is to run via, APIs - and call the API via - bash task.

Here is the script

param (
    [string]$token="",
    [string]$collection="",
    [string]$projectName ="",
    [int] $planIdStatic = ,
    [int] $suiteIdStatic = ,
    [int] $testcaseID = 


)

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user, $token)))
$response = Invoke-RestMethod "https://dev.azure.com/$collection/$projectName/_apis/test/plans?api-version=5.0" -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
foreach( $val in $response.value)
{

#PLAN ID 
Write-Host $val.id
 # $planId = [convert]::ToInt32($val.id)
 [int] $planId = $planIdStatic
  $suites = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suites -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 #define Suite ID

 [int] $suiteId = $suiteIdStatic
 $suitename = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suitename -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}


 #Define TestCaseID

 [int] $tcID = $testcaseID

 $tc = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?testCaseId=$tcID&api-version=5.0"
 $testcaseapi = Invoke-RestMethod $tc -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 
 #Define PointID

[int] $pointID =    $testcaseapi.value[0].id 
 $runName = "Test RUN Setup"

 #PayLoad

 $payload = @"
 
 {
   "name":"$runName",
   "plan":{
      "id":"$planId"
   },
   "pointIds":[
      "$pointID"
   ]
}
"@;

#Initiate the RUN

$tcRun = "https://dev.azure.com/$collection/$projectName/_apis/test/runs?api-version=5.0"
 $testRun = Invoke-RestMethod $tcRun -Method 'POST'  -ContentType "application/json"  -Body $payload -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 
Write-Host "Test Run Status is ...." 
}

From the blog - https://ghoshasish99.medium.com/integrate-your-test-automation-framework-with-azure-test-plan-a3230ab0f1da

Unfortunately it creates the test run - but it wont run any testcase it just shows as in-progress forever, and that's all - (Note - Test Plan is associated with Build and Release)

I suspect the script needs payload- body with build and release pipeline & release stage (Assume my release has DEV - QA and PROD) stages.

Answer 1

In addition to using Rest API to run test run, you also need to use Release API to run release and need to update the running release information to test run.

Here are the following steps:

Step1: You could define a variable in Release Pipeline.

Step2: Add this variable in VSTest Task:

Note: The test run id needs to correspond to the release one-to-one, and then the status of the test run will be updated.

Here is an example:

param (
    [string]$token="",
    [string]$collection="",
    [string]$projectName ="",
    [int] $planIdStatic =947 ,
    [int] $suiteIdStatic =1086 ,
    [int] $testcaseID =79,
    [int] $releasedefinitionid =96,
    [int] $buildid =62903


)

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user, $token)))
$response = Invoke-RestMethod "https://dev.azure.com/$collection/$projectName/_apis/test/plans?api-version=5.0" -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
foreach( $val in $response.value)
{

#PLAN ID 
Write-Host $val.id
 # $planId = [convert]::ToInt32($val.id)
 [int] $planId = $planIdStatic
  $suites = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suites -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 #define Suite ID

 [int] $suiteId = $suiteIdStatic
 $suitename = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?api-version=5.0"
 $listofSuites = Invoke-RestMethod $suitename -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}


 #Define TestCaseID

 [int] $tcID = $testcaseID

 $tc = "https://dev.azure.com/$collection/$projectName/_apis/test/plans/$planId/suites/$suiteId/points?testCaseId=$tcID&api-version=5.0"
 $testcaseapi = Invoke-RestMethod $tc -Method 'GET' -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)}
 
 #Define PointID

[int] $pointID =    $testcaseapi.value[0].id 
 $runName = "Test RUN Setup"

 #PayLoad

 $payload = @"
 
 {
   "name":"test",
    "automated":true,
    "pointIds":[$pointID],
    "state":"NotStarted",
    "dtlTestEnvironment":{"id":"vstfs://dummy"},
    "plan":{"id":"$planId"},
    "filter":{"sourceFilter":"*.dll","testCaseFilter":""}
}
"@;

#Initiate the RUN

$tcRun = "https://dev.azure.com/$collection/$projectName/_apis/test/runs?api-version=5.0"
 $testRun = Invoke-RestMethod $tcRun -Method 'POST'  -ContentType "application/json"  -Body $payload -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 
Write-Host "Test Run Status is ...."

$testrunid = $testRun.id

echo $testrunid


$url = "https://vsrm.dev.azure.com/$collection/$projectName/_apis/Release/definitions/$($releasedefinitionid)?api-version=5.0-preview.3"
Write-Host "URL: $url"
$pipeline = Invoke-RestMethod -Uri $url -Method Get  -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 

Write-Host "Pipeline = $($pipeline | ConvertTo-Json -Depth 100)"


$pipeline.variables.testrunid.value = "$testrunid"

####****************** update the modified object **************************
$json = @($pipeline) | ConvertTo-Json -Depth 99


$updatedef = Invoke-RestMethod -Uri $url -Method Put -Body $json -ContentType "application/json" -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 



$releaseRunurl ="https://vsrm.dev.azure.com/$collection/$projectName/_apis/Release/releases?api-version=6.1-preview.8"

 $releasebody = @"
 
 {
     "definitionId": $releasedefinitionid

}
"@;

$ReleaseRun = Invoke-RestMethod $releaseRunurl -Method 'POST'  -ContentType "application/json"  -Body $releasebody -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 

$Releaseid=$ReleaseRun.id

echo $Releaseid

$ReleaseEnvID=$ReleaseRun.environments.id

echo $ReleaseEnvID

$updateTestrun="https://dev.azure.com/$collection/$projectName/_apis/test/Runs/$($testrunid)?api-version=6.1-preview.3"

$updatebody = @"
 
 {
    "build":
    {
        "id":"$buildid"
    },
    "releaseEnvironmentUri":"vstfs:///ReleaseManagement/Environment/$ReleaseEnvID","releaseUri":"vstfs:///ReleaseManagement/Release/$Releaseid"

}
"@;


$UpdateRun = Invoke-RestMethod $updateTestrun -Method 'PATCH'  -ContentType "application/json"  -Body $updatebody -Headers @{Authorization = ("Basic {0}" -f $base64AuthInfo)} 


 

}

Result:

When the release is completed, the status of the test run will be updated