Azure Cloud Service Extended deployment via dev ops pipeline fails

Question

I created a sample .net sdk web api application in VS. I also created Azure Cloud Service extended in portal and updated it with the package from VS. Initially, I did not have Virtual Network settings in the configuration files, but I later added them. I am able to publish successfully from VS using publish option. However, the deployment fails when triggered via release pipeline in dev ops. The code is in git hub. Every check in will trigger a build and release from dev ops. The error is shown below. The target service is the azure cloud service extended that was already created in the portal. In the debug output, I see that the cmd Get-AzureDeployment -ServiceName kgcloudservicefordevopstesting -Slot Production did not find the resource and hence the script goes into creation of the service (rather than update) which seems to throw this issue. I can see the resource using Az powershell cmd, however. I am using Azure Cloud Service deployment helper in the pipeline. It only shows my classic storage account but not newer storage accounts. I am wondering if I should be using some other helper but can't find anything similar. Any help would be appreciated.

PS C:\WINDOWS\system32> Get-AzCloudService

ResourceGroupName Name Location ProvisioningState

---

WESTUS kgcloudservicefordevopstesting westus Succeeded

Debug lines from pipeline: 021-12-10T03:26:06.3881535Z ##[command]Get-AzureDeployment -ServiceName kgcloudservicefordevopstesting -Slot Production -ErrorAction SilentlyContinue -ErrorVariable azureDeploymentError 2021-12-10T03:26:06.4024994Z ##[debug]3:26:06 AM - Begin Operation: Get-AzureDeployment 2021-12-10T03:26:06.9384711Z ##[debug]3:26:06 AM - Completed Operation: Get-AzureDeployment 2021-12-10T03:26:07.0894610Z ##[debug]Microsoft.WindowsAzure.Commands.Common.ComputeCloudException: ResourceNotFound: No deployments were found. 2021-12-10T03:26:07.0928109Z ##[debug]OperationID : '4278a00d915739b7bdd6579aef8d3191' ---> Hyak.Common.CloudException: ResourceNotFound: No deployments were found. 2021-12-10T03:26:07.0968346Z ##[debug] at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) 2021-12-10T03:26:07.1001646Z ##[debug] at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task) 2021-12-10T03:26:07.1033857Z ##[debug] at Microsoft.WindowsAzure.Management.Compute.DeploymentOperationsExtensions.GetBySlot(IDeploymentOperations operations, String serviceName, DeploymentSlot deploymentSlot) 2021-12-10T03:26:07.1071183Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.ServiceManagementBaseCmdlet.ExecuteClientActionNewSM[TResult](Object input, String operationDescription, Func1 action, Func3 contextFactory) in d:\workspace\powershell-sign\src\ServiceManagement\Services\Commands.Utilities\Common\ServiceManagementBaseCmdlet.cs:line 152 2021-12-10T03:26:07.1097514Z ##[debug] --- End of inner exception stack trace --- 2021-12-10T03:26:07.1126379Z ##[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.ServiceManagementBaseCmdlet.ExecuteClientActionNewSM[TResult](Object input, String operationDescription, Func1 action, Func3 contextFactory) in d:\workspace\powershell-sign\src\ServiceManagement\Services\Commands.Utilities\Common\ServiceManagementBaseCmdlet.cs:line 163 2021-12-10T03:26:07.1182003Z ##[command]New-AzureDeployment -ServiceName kgcloudservicefordevopstesting -Package D:\a\r1\a_kganesan1.MyCloudServiceExtended\drop\s\AzureCloudService1\AzureCloudService1\bin\Release\app.publish\AzureCloudService1.cspkg -Configuration D:\a\r1\a_kganesan1.MyCloudServiceExtended\drop\s\AzureCloudService1\AzureCloudService1\bin\Release\ServiceConfiguration.cscfg -Slot Production -Label 20211210.7 12/10/2021 03:26:06 -ExtensionConfiguration 2021-12-10T03:26:09.7380161Z ##[debug]3:26:09 AM - Begin Operation: New-AzureDeployment 2021-12-10T03:26:10.2564315Z ##[debug]Leaving D:\a_tasks\AzureCloudPowerShellDeployment_2ca8fe15-42ea-4b26-80f1-e0738ec17e89\1.184.0\Publish-AzureCloudDeployment.ps1. 2021-12-10T03:26:10.2667166Z ##[debug]Caught exception from task script. 2021-12-10T03:26:10.2714719Z ##[debug]Error record: 2021-12-10T03:26:10.4139673Z ##[debug]New-AzureDeployment : ConflictError: Instead of creating Cloud Services (classic), you should now create Cloud Services (extended support) using Azure Resource Manager. Learn more at - https://aka.ms/cloudservicesretirement. 2021-12-10T03:26:10.4163429Z ##[debug]OperationID : '0c1929d2629e3af9a9c0c70f04c5be26' 2021-12-10T03:26:10.4214192Z ##[debug]At D:\a_tasks\AzureCloudPowerShellDeployment_2ca8fe15-42ea-4b26-80f1-e0738ec17e89\1.184.0\Publish-AzureCloudDeployment.ps1:115 char:32 2021-12-10T03:26:10.4235990Z ##[debug]+ ... eployment = New-AzureDeployment -ServiceName $ServiceName -Package $s ...

Answer 1

Got this working with GitHub actions, if anyone else needs that. Main bits heavily inspired by @gplwhite's other answer.

name: Deploy Azure Cloud Service

on:
  push:
    branches:
      - main

jobs:
  build-and-deploy:
    runs-on: windows-latest

    env:
      SERVICE_NAME: "<your-service>"
      RESOURCE_GROUP_NAME: "<your-rg>"
      LOCATION: "Australia Southeast"
      CLOUD_SERVICE_NAME: "<your-service>"
      CONFIG_NAME: "Cloud"
      BUILD_CONFIGURATION: "Release"
      STORAGE_ACCOUNT: "<your-stroageaccount>"      

    steps:
    - name: 'Checkout Code'
      uses: actions/checkout@v3

    - name: 'Install Azure Powershell (Az) module'
      shell: pwsh
      run: |
        Install-Module -Name Az -Force -AllowClobber -Scope CurrentUser
        Import-Module Az    

    - name: 'Uninstall AzureRM to avoid conflicts'
      shell: pwsh
      run: Uninstall-AzureRm

    - name: 'Login to Azure via Service Principal'
      shell: pwsh
      run: |
        $clientId = "${{ secrets.AZURE_CLIENT_ID }}"
        $clientSecret = "${{ secrets.AZURE_CLIENT_SECRET }}"
        $tenantId = "${{ secrets.AZURE_TENANT_ID }}"
        $subscriptionId = "${{ secrets.AZURE_SUBSCRIPTION_ID }}"
        $securePassword = ConvertTo-SecureString $clientSecret -AsPlainText -Force
        $credential = New-Object System.Management.Automation.PSCredential($clientId, $securePassword)
        Connect-AzAccount -ServicePrincipal -TenantId $tenantId -Credential $credential -SubscriptionId $subscriptionId

    - name: 'Restore NuGet Packages'
      run: nuget restore ${{ env.CLOUD_SERVICE_NAME }}.sln

    - name: 'Build and Package Cloud Service'
      run: |
       & "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\MSBuild.exe" ${{ env.CLOUD_SERVICE_NAME }}.ccproj /p:Configuration=${{ secrets.BUILD_CONFIGURATION }} /t:Publish

    - name: Set VERSION environment variable
      shell: pwsh
      run: |
        $version = (100 + $env:GITHUB_RUN_NUMBER).ToString() + ".0.0"
        echo "VERSION=$version" >> $env:GITHUB_ENV      

    - name: 'Deploy to Azure Cloud Service'
      shell: pwsh
      run: |
        $cloudServiceName = "${{ env.SERVICE_NAME }}"
        $resourceGroupName = "${{ env.RESOURCE_GROUP_NAME }}"
        $storageAccountName = "${{ env.STORAGE_ACCOUNT }}"
        $storageContainerName = "deploy"

        $buildArtifactsPath = "bin\app.publish"
        $configName = "ServiceConfiguration.${{ env.CONFIG_NAME }}.cscfg"
        $packageName = "${{ env.CLOUD_SERVICE_NAME }}.cspkg"

        Write-Host "Finding existing service $cloudServiceName in Resource Group $resourceGroupName"

        $svc = Get-AzCloudService -ResourceGroup $resourceGroupName -CloudServiceName $cloudServiceName
        if (!$svc)
        {   
            throw "Failed to find existing cloud service $cloudServiceName"
        }

        Write-Host "Checking if storage account $storageAccountName exists"

        $storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroupName -Name $storageAccountName -ErrorAction SilentlyContinue
        if (!$storageAccount)
        {
            throw "Failed to find existing storage account $storageAccountName"
        }

        Write-Host "Checking if storage container $storageContainerName exists"
        $container = Get-AzStorageContainer -Name $storageContainerName -Context $storageAccount.Context -ErrorAction SilentlyContinue

        if (!$container)
        {
            Write-Host "Storage account container not found. Creating container $storageContainerName"
            $container = New-AzStorageContainer -Name $storageContainerName -Context $storageAccount.Context -Permission Off 
        }

        $tokenStartTime = Get-Date 
        $tokenEndTime = $tokenStartTime.AddYears(1) 

        Write-Host "Uploading service configuration file to blob storage"

        $cscfgBlob = Set-AzStorageBlobContent -File "$buildArtifactsPath/$configName" -Container $storageContainerName -Blob "cloudservice.cscfg" -Context $storageAccount.Context -Force
        $cscfgToken = New-AzStorageBlobSASToken -Container $storageContainerName -Blob $cscfgBlob.Name -Permission r -StartTime $tokenStartTime -ExpiryTime $tokenEndTime -Context $storageAccount.Context 
        $cscfgUrl = $cscfgBlob.ICloudBlob.Uri.AbsoluteUri + "?" + $cscfgToken 

        Write-Host "Service configuration uploaded."

        Write-Host "Uploading service package file to blob storage"

        $cspkgBlob = Set-AzStorageBlobContent -File "$buildArtifactsPath/$packageName" -Container $storageContainerName -Blob "cloudservice.cspkg" -Context $storageAccount.Context -Force
        $cspkgToken = New-AzStorageBlobSASToken -Container $storageContainerName -Blob $cspkgBlob.Name -Permission r -StartTime $tokenStartTime -ExpiryTime $tokenEndTime -Context $storageAccount.Context 
        $cspkgUrl = $cspkgBlob.ICloudBlob.Uri.AbsoluteUri + "?" + $cspkgToken 

        Write-Host "Service package uploaded."

        Write-Host "Updating existing service $cloudServiceName with new configuration and executable package"

        New-AzCloudService `
        -Name $svc.Name `
        -ResourceGroupName $svc.ResourceGroupName `
        -Location $svc.Location `
        -ConfigurationUrl $cscfgUrl `
        -PackageUrl $cspkgUrl `
        -UpgradeMode Auto `
        -RoleProfile $svc.RoleProfile `
        -NetworkProfile $svc.NetworkProfile `
        -OSProfile $svc.OSProfile `
        -ExtensionProfile $svc.ExtensionProfile `
        -Tag @{"DeploymentLabel" = "$env:VERSION"}

        Write-Host "Cloud Service Updated"

Youll need to make sure you setup an Azure Service principal with Contributor access to the resource group.

Hope that helps others.

Answer 2

Here's a powershell script I use for updating an existing Cloud Service (Extended Support). I just pop this in an Azure Pipelines Powershell Task.

Note: Assumes the CloudService has already been created and deployed as it uses the existing Os/Network/Role/Extension definitions. Also assumes the storage account exists and lives in the same resource group as the Cloud Service.

$cloudServiceName = "<Name of an existing Cloud Service Extended Support service>"
$resourceGroupName = "<Name of the resource group containing the Cloud Service>"
$storageAccountName = "<Name of a storage account where temporary deployment files are stored>"
$storageContainerName = "<Name of the storage container where temporary deployment files are stored>"

$buildArtifactsPath = "<Path to build artifacts downloaded by Pipeline>"
$configName = "<File name of cloud service config file *.cscfg>"
$packageName = "<File name of cloud service package file *.cspkg>"


Write-Host "Finding existing service $cloudServiceName in Resource Group $resourceGroupName"

$svc = Get-AzCloudService -ResourceGroup $resourceGroupName -CloudServiceName $cloudServiceName
if (!$svc)
{   
    throw "Failed to find existing cloud service $cloudServiceName"
}


Write-Host "Checking if storage account $storageAccountName exists"

$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroupName -Name $storageAccountName -ErrorAction SilentlyContinue
if (!$storageAccount)
{
    throw "Failed to find existing storage account $storageAccountName"
}

Write-Host "Checking if storage container $storageContainerName exists"
$container = Get-AzStorageContainer -Name $storageContainerName -Context $storageAccount.Context -ErrorAction SilentlyContinue

if (!$container)
{
    Write-Host "Storage account container not found. Creating container $storageContainerName"
    $container = New-AzStorageContainer -Name $storageContainerName -Context $storageAccount.Context -Permission Off 
}


$tokenStartTime = Get-Date 
$tokenEndTime = $tokenStartTime.AddYears(1) 


Write-Host "Uploading service configuration file to blob storage"

$cscfgBlob = Set-AzStorageBlobContent -File "$(buildArtifactsPath)/$(configName)" -Container $storageContainerName -Blob "cloudservice.cscfg" -Context $storageAccount.Context -Force
$cscfgToken = New-AzStorageBlobSASToken -Container $storageContainerName -Blob $cscfgBlob.Name -Permission r -StartTime $tokenStartTime -ExpiryTime $tokenEndTime -Context $storageAccount.Context 
$cscfgUrl = $cscfgBlob.ICloudBlob.Uri.AbsoluteUri + $cscfgToken 

Write-Host "Service configuration uploaded."

Write-Host "Uploading service package file to blob storage"

$cspkgBlob = Set-AzStorageBlobContent -File "$(buildArtifactsPath)/$(packageName)" -Container $storageContainerName -Blob "cloudservice.cspkg" -Context $storageAccount.Context -Force
$cspkgToken = New-AzStorageBlobSASToken -Container $storageContainerName -Blob $cspkgBlob.Name -Permission r -StartTime $tokenStartTime -ExpiryTime $tokenEndTime -Context $storageAccount.Context 
$cspkgUrl = $cspkgBlob.ICloudBlob.Uri.AbsoluteUri + $cspkgToken 

Write-Host "Service package uploaded."


Write-Host "Updating existing service $cloudServiceName with new configuration and executable package"


New-AzCloudService `
-Name $svc.Name `
-ResourceGroupName $svc.ResourceGroupName `
-Location $svc.Location `
-ConfigurationUrl $cscfgUrl `
-PackageUrl $cspkgUrl `
-UpgradeMode Auto `
-RoleProfile $svc.RoleProfile `
-NetworkProfile $svc.NetworkProfile `
-OSProfile $svc.OSProfile `
-ExtensionProfile $svc.ExtensionProfile `
-Tag @{"DeploymentLabel" = "$(Build.BuildNumber)"}


Write-Host "Cloud Service Updated"

Someone could probably turn this into a reusable custom task without too much effort. https://learn.microsoft.com/en-us/azure/devops/extend/develop/add-build-task?view=azure-devops

Answer 3

Microsoft is retiring Cloud Services (Classic) as described at https://aka.ms/cloudservicesretirement.

As part of that retirement they seem to have recently blocked deploying to newly-created Cloud Services (Classic) instances.

I attempted to deploy a Cloud Service via Visual Studio as you mentioned, and was able to do it via Right-Click -> "Publish (Extended Support)", but not via Right-Click -> "Publish".

This is because the Extended Support option creates a different kind of Cloud Service which will be supported for the longer term. The "Publish" option creates a Classic cloud service.

I think the summary is: you can no longer create a Classic cloud service and deploy to it; if you're building a new Cloud Service, you should use the "Cloud Services (Extended Support)" option instead, described here in the Azure doco.