CODEWITHSUNDEEP
azureazure-devopsazure-powershell
SJB
SJB

Reputation: 197

Azure DevOps Pipeline Azure Blob Storage upload file 403 Forbidden Exception

Summary

I'm creating a CI/CD provisioning pipeline for a new Azure Storage Account within an Azure DevOps Pipeline and attempting to upload some files into the Blob Storage using AzCopy running from an Azure Powershell task in the pipeline.

The Error

The script runs successfully from my local machine but when running in the Azure DevOps pipeline I get the following error (ErrorDateTime is just an obfuscated ISO 8601 formatted datetime):

Assumptions

What I've tried...

I'm using the following line within the Azure Powershell Task. I'm happy with the values because everything works when "All networks" or my IP Address is enabled and I run locally.

.\AzCopy.exe /Source:$SourcePath /Dest:$blobUrlDir /DestKey:$key /Pattern:$FilenamePattern /Y

Any thoughts or guidance would be appreciated.

Thanks,

SJB

Upvotes: 6

Views: 8887

Answers (4)

B8ightY
B8ightY

Reputation: 567

In my case, the Service Principal from Azure Subscription selected in pipeline needed to have role of Storage Blob Data Contributor for the desired Storage Account where I wanted to copy files.

To do this, follow these steps:

  1. Open the DevOps pipeline, find the Azure subscription field and click on "Manage" button next to it
  2. Then click on "Manage Service Principal" and note the Display name (optionally changing it to something unique -- as all the service principals for service connections in a project have the same default name which can be confusing for the next step)
  3. Open the Storage Resource (Account or Container) in the Azure Portal, then select "Access Control (IAM)" in the blade on the left, then click "Add a role assignment" and add the "Storage Blob Data Contributor" role to the Service Principal you noted in previous step

If the "Add a role assignment" is disabled or greyed out, you need to ask your administrator to grant that access.

This solution was taken and improved from this GitHub comment.

Upvotes: 0

Matt Wanchap
Matt Wanchap

Reputation: 909

People seem to be getting mixed results in this github issue, but the AzureFileCopy@4 task works (at least for us) after adding the "Storage Blob Data Contributor" role to the ARM connection's service principal (to the storage account itself). The below is the only necessary step in a pipeline that deploys a repo as a static website in a blob container:

- task: AzureFileCopy@4
  displayName: 'Copy files to blob storage: $(storageName)'
  inputs:
    SourcePath: '$(build.sourcesDirectory)'
    Destination: AzureBlob
    storage: $(storageName)
    ContainerName: $web
    azureSubscription: 'ARM Connection goes here' # needs a role assignment before it'll work

(Of course, if you're using Azure CDN like we are, the next step is to clear the CDN endpoint's cache, but that has nothing to do with the blob storage error)

Upvotes: 6

Remco Brosky
Remco Brosky

Reputation: 1

Have you considered using the Azure DevOps Task "Azure File Copy" instead of a powershell script? see: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-file-copy?view=azure-devops

Upvotes: -2

SJB
SJB

Reputation: 197

After doing further research I noticed the following raised issue - that Azure DevOps isn't considered a trusted Microsoft Service from a Storage Account perspective.

My temporary workaround is to:

  • Setting the DefaultAction to Allow, thereby allowing "All networks access".
  • Setting the DefaultAction to Deny after the copy action ensured my VNet rules were being enforced again.
Try
{
    Update-AzureRmStorageAccountNetworkRuleSet -ResourceGroupName "$ResourceGroupName" -Name "$StorageAccountName" -DefaultAction Allow
    .\AzCopy.exe /Source:$SourcePath /Dest:$blobUrlDir /DestKey:$key /Pattern:$FilenamePattern /Y
}
Catch
{
    #Handle errors...
}
Finally
{
    Update-AzureRmStorageAccountNetworkRuleSet -ResourceGroupName "$ResourceGroupName" -Name "$StorageAccountName" -DefaultAction Deny
}

Thanks,

SJB

Upvotes: 5

Related Questions