CODEWITHSUNDEEP
azureazure-devopsazure-yaml-pipelines
user2503480
user2503480

Reputation: 193

Why my multi stage full yaml azure devops pipeline breaks when moved to template?

I have a multi stage full yaml pipeline in azure devops services. The different stages consist of a build stage, a Dev stage (Deploys azure resources to an azure resource group) and a UAT stage (Deploys azure resources to a different resource group). Everything works great. However I needed to use templates because I want to reuse the steps. After moving the steps to a template, the dev stage keeps working but the UAT stage gives authorization error. It is worth mentioning, dev and uat stages use different service connections targeting different subscriptions. But why will it work when I dont use templates and simply moving the steps to a template file all of a sudden it doesn't work. Is this a known bug?

This is the error I get when using the template approach:

 | The client '9e5cc21a-bb38-46b9-a16a-289fbbf9c8b9' with object
     | id '9e5cc21a-bb38-46b9-a16a-289fbbf9c8b9' does not have
     | authorization to perform action
     | 'Microsoft.Resources/subscriptions/resourceGroups/resources/read' over scope '/subscriptions/7041f5ba-1040-4989-8e48-497b3b826d01/resourceGroups/Resource-Group-Test-A' or the scope is invalid. If access was recently granted, please refresh your credentials. StatusCode: 403 ReasonPhrase: Forbidden OperationID : 464f577e-6617-4bed-9a14-1f7487b5f209

This is the pipeline without using templates (works perfect!).

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master

variables:
- group: 'Dev ARMT SFTP Connection'

resources:
  repositories:
    - repository: templates
      type: git
      name: CommonTasks

stages: 
- stage: Build
  jobs:
    - job: Build
      pool:
        vmImage: 'ubuntu-latest'
      steps:
      - task: printAllVariables@1
      - task: CopyFiles@2
        inputs:
          SourceFolder: '$(System.DefaultWorkingDirectory)'
          Contents: 'ARM-Templates/**'
          TargetFolder: '$(Build.ArtifactStagingDirectory)'
      - task: PublishBuildArtifacts@1
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)'
          ArtifactName: 'drop'
          publishLocation: 'Container'
      - task: PowerShell@2
        inputs:
          targetType: 'inline'
          script: |
            # Write your PowerShell commands here.
            
            Write-Host "Build Completed..."
            $workingdir = "$(Build.ArtifactStagingDirectory)"
            Write-Host $workingdir
            $fcontent = Get-ChildItem -Path $workingdir
            Write-Host $fcontent

    
- stage: Dev
  jobs:
  - job: Dev
    pool:
        vmImage: 'ubuntu-latest'
    steps:
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'specific'
        project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
        pipeline: '30'
        buildVersionToDownload: 'latest'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
    - task: FileTransform@1
      displayName: 'ARMT SFTP Connection - Parameters File Transform '
      inputs:
        folderPath: '$(System.ArtifactsDirectory)/SftpConnection/drop'
        fileType: json
        targetFiles: 'ARM-Templates/parameters.json'
    - task: AzurePowerShell@4
      displayName: 'ARMT SFTP Connection - Check Resource Existance'
      inputs:
        azureSubscription: 'Starwood-DT-DEV-ServiceConnection'
        ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
        ScriptArguments: '-resourceGroupName Starwood-DT-DEV -resourceName $(parameters.sftp_name.value)'
        azurePowerShellVersion: LatestVersion
    - task: AzureResourceGroupDeployment@2
      displayName: 'ARMT SFTP Connection - Deploy'
      inputs:
        azureSubscription: 'Starwood-DT-DEV-ServiceConnection'
        resourceGroupName: 'Starwood-DT-DEV'
        location: 'East US'
        csmFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
        csmParametersFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
      condition: eq(variables['deployresource'],'true')

- stage: GroupATest
  jobs:
  - job: GroupATest
    pool:
        vmImage: 'ubuntu-latest'
    steps:
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'specific'
        project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
        pipeline: '30'
        buildVersionToDownload: 'latest'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
    - task: FileTransform@1
      displayName: 'ARMT SFTP Connection - Parameters File Transform '
      inputs:
        folderPath: '$(System.ArtifactsDirectory)/SftpConnection/drop'
        fileType: json
        targetFiles: 'ARM-Templates/parameters.json'
    - task: AzurePowerShell@4
      displayName: 'ARMT SFTP Connection - Check Resource Existance'
      inputs:
        azureSubscription: 'Resource-Group-Test-A'
        ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
        ScriptArguments: '-resourceGroupName Resouce-Group-Test-A -resourceName $(parameters.sftp_name.value)'
        azurePowerShellVersion: LatestVersion

    - task: AzureResourceGroupDeployment@2
      displayName: 'ARMT SFTP Connection - Deploy'
      inputs:
        azureSubscription: 'Resource-Group-Test-A'
        resourceGroupName: 'Resouce-Group-Test-A'
        location: 'East US'
        csmFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
        csmParametersFile: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'
      condition: eq(variables['deployresource'],'true')

When moving to steps template (then i get the error):

 # Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master


resources:
  repositories:
    - repository: templates
      type: git
      name: CommonTasks

stages: 
- stage: Build
  jobs:
    - job: Build
      pool:
        vmImage: 'ubuntu-latest'
      steps:
      - task: printAllVariables@1
      - task: CopyFiles@2
        inputs:
          SourceFolder: '$(System.DefaultWorkingDirectory)'
          Contents: 'ARM-Templates/**'
          TargetFolder: '$(Build.ArtifactStagingDirectory)'
      - task: PublishBuildArtifacts@1
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)'
          ArtifactName: 'drop'
          publishLocation: 'Container'
      - task: PowerShell@2
        inputs:
          targetType: 'inline'
          script: |
            # Write your PowerShell commands here.
            
            Write-Host "Build Completed..."
            $workingdir = "$(Build.ArtifactStagingDirectory)"
            Write-Host $workingdir
            $fcontent = Get-ChildItem -Path $workingdir
            Write-Host $fcontent

    
- stage: Dev
  variables:
  - group: 'Dev ARMT SFTP Connection'
  jobs:
  - job: Dev
    pool:
        vmImage: 'ubuntu-latest'
    steps:
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'specific'
        project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
        pipeline: '30'
        buildVersionToDownload: 'latest'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
    - template: YamlTemplate/azure-resource-deploy.yml@templates
      parameters:
        dropLocation: '$(System.ArtifactsDirectory)/SftpConnection/drop'
        transformTargetPath: 'ARM-Templates/parameters.json'
        resourceName: $(parameters.sftp_name.value)
        resourceGroupName: 'Starwood-DT-DEV'
        azureServiceConnectionName: 'Starwood-DT-DEV-ServiceConnection'
        resourceLocation: 'East US'
        armtTemplateFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
        armtParemeterFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'

- stage: GroupATest
  variables:
  - group: 'GroupA ARMT SFTP Connection'
  jobs:
  - job: GroupATest
    pool:
        vmImage: 'ubuntu-latest'
    steps: 
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'specific'
        project: '530cfd4b-51b8-4237-b2fa-f296a4cba29d'
        pipeline: '30'
        buildVersionToDownload: 'latest'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/CommonTasks'
    - task: DownloadBuildArtifacts@0
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'drop'
        downloadPath: '$(System.ArtifactsDirectory)/SftpConnection'
    - template: YamlTemplate/azure-resource-deploy.yml@templates
      parameters:
        dropLocation: '$(System.ArtifactsDirectory)/SftpConnection/drop'
        transformTargetPath: 'ARM-Templates/parameters.json'
        resourceName: $(parameters.sftp_name.value)
        resourceGroupName: 'Resource-Group-Test-A'
        azureServiceConnectionName: 'Resource-Group-Test-A'
        resourceLocation: 'East US'
        armtTemplateFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/template.json'
        armtParemeterFilePath: '$(System.ArtifactsDirectory)/SftpConnection/drop/ARM-Templates/parameters.json'

This the template structure Parent template - (azure-resource-deploy.yml):

parameters:
- name: dropLocation
  type: string
  displayName: 'Artifact path to drop location'
  default: ''
- name: transformTargetPath
  type: string
  default: ''
- name: resourceName
  type: string
  default: ''
- name: resourceGroupName
  type: string
  default: ''
- name: azureServiceConnectionName
  type: string
  default: ''
- name: resourceLocation
  type: string
  default: ''
- name: armtTemplateFilePath
  type: string
  default: ''
- name: armtParemeterFilePath
  type: string
  default: ''


steps:
- template: armt-parameter-file-transform.yml
  parameters:
   transformStepDisplayName: ''
   folderPath: ${{parameters.dropLocation}}
   targetFile: ${{parameters.transformTargetPath}}

- template: azure-resource-check.yml
  parameters:
   resourceName: ${{parameters.resourceName}}
   resourceGroupName: ${{parameters.resourceGroupName}}
   azureServiceConnectionName: ${{parameters.azureServiceConnectionName}}


- template: armt-deploy.yml
  parameters:
   resourceName: ${{parameters.resourceName}}
   resourceGroupName: ${{parameters.resourceGroupName}}
   resourceLocation: ${{parameters.resourceLocation}}
   azureServiceConnectionName: ${{parameters.azureServiceConnectionName}}
   templateFilePath: ${{parameters.armtTemplateFilePath}}
   parametersFilePath: ${{parameters.armtParemeterFilePath}}

Templates referenced from parent:

(armt-parameter-file-transform.yml)

parameters:
- name: transformStepDisplayName
  type: string
  displayName: 'Display name for this step'
  default: ''
- name: folderPath
  type: string
  displayName: 'Path to drop location'
  default: ''
- name: targetFile
  type: string
  displayName: 'Path to paremeter file relative to drop location.'
  default: ''

steps:
- task: FileTransform@1
  displayName: ${{parameters.transformStepDisplayName}}
  inputs:
    folderPath: ${{parameters.folderPath}}
    fileType: json
    targetFiles: ${{parameters.targetFile}}

(azure-resource-check.yml)

parameters:
- name: resourceName
  type: string
- name: resourceGroupName
  type: string
- name: azureServiceConnectionName
  type: string

steps:
- script: echo Echo -resourceGroupName ${{ parameters.resourceGroupName }} -resourceName ${{ parameters.resourceName }}
- task: AzurePowerShell@4
  displayName: '${{ parameters.resourceName }} - Checking Resource Existance'
  inputs:
    azureSubscription: '${{ parameters.azureServiceConnectionName }}'
    ScriptPath: '$(System.ArtifactsDirectory)/CommonTasks/drop/AzurePowerShell/ResourceExistance.ps1'
    ScriptArguments: '-resourceGroupName ${{ parameters.resourceGroupName }} -resourceName ${{ parameters.resourceName }}'
    azurePowerShellVersion: LatestVersion

(armt-deploy.yml)

parameters:
- name: resourceName
  type: string
- name: resourceGroupName
  type: string
- name: resourceLocation
  type: string
- name: azureServiceConnectionName
  type: string
- name: templateFilePath
  type: string
- name: parametersFilePath
  type: string

steps:
- task: AzureResourceGroupDeployment@2
  displayName: 'ARMT Deploy - ${{parameters.resourceName}}'
  inputs:
    azureSubscription: ${{parameters.azureServiceConnectionName}}
    resourceGroupName: ${{parameters.resourceGroupName}}
    location: ${{parameters.resourceLocation}}
    csmFile: ${{parameters.templateFilePath}}
    csmParametersFile: ${{parameters.parametersFilePath}}
  condition: eq(variables['deployresource'],'true')

Upvotes: 0

Views: 804

Answers (1)

user2503480
user2503480

Reputation: 193

This was just caused by a typo. The actual name of the resource group is Resouce-Group-Test-A, I missed the r when naming this resource group. The error does not happen in the non-templated version because when you pick the resource group you do it from a drop down list, with the correct prepopulated names, so no chance to make a mistake. However in a full yaml pipeline you have to type it in, and thats where the typo occured. If there is anything to be learned here. Pay close attention when typing resource names, errors might not be descriptive enough. Sorry If wasted anybodys precious time.

Upvotes: 0

Related Questions