Azure ARM Template Keyvault Resources keeps removing other access policies

Question

I created an ARM template to deploy an Azure WebApp that is using Managed Service Identity authentication with KeyVault for secrets. So the ARM template creates the WebApp resource and enables MSI, and also creates the KeyVault resource and add the WebApp tenantid and objectid to the accessPolicies, however, the ARM template also removes all other existing access policies from my Keyvault.

Is there a way to do an incremental deployment of the access policies, so that I don't have to add back users to the KeyVault access policies after deploymment?

{
  "type": "Microsoft.KeyVault/vaults",
  "name": "[parameters('ICMODSKeyVaultName')]",
  "apiVersion": "2016-10-01",
  "location": "[resourceGroup().location]",
  "properties": {
    "sku": {
      "family": "A",
      "name": "standard"
    },
    "tenantId": "[reference(variables('identityResourceId'), '2015-08-31-PREVIEW').tenantId]",
    "accessPolicies": [
      {
        "tenantId": "[reference(variables('identityResourceId'), '2015-08-31-PREVIEW').tenantId]",
        "objectId": "[reference(variables('identityResourceId'), '2015-08-31-PREVIEW').principalId]",
        "permissions": {
          "secrets": [
            "get"
          ]
        }
      }
    ],
    "enabledForDeployment": true,
    "enabledForTemplateDeployment":  true
  },
  "dependsOn": [
    "[concat('Microsoft.Web/sites/', parameters('AppName'))]"
  ]
}

Answer 1

I've managed to create a solution to first read the AccessPolicies, if the KeyVault resource exists, and then use these as a parameter. For this I used the following scripts. Please note: that you cannot merge this logic into one file, otherwise you will get errors.

In this example, the name of the KeyVault resource is: 'MyKeyVault' and my user-assigned managed identity is: 'MyUserAssignedManagedIdentity'. This Identity has the reader role on the ResourceGroup in Azure of the KeyVault resource.

These scripts work in the case that the KeyVault doesn't exist and when it already exists. When it exisits it preserves the AccessPolicies.

File 1: MainDeployment.bicep (Your starting point script)

var location = resourceGroup().location
var keyVaultName = 'MyKeyVault'

module keyVaultModule 'KeyVaultResourcePreservingAccessPolicies.bicep' = {
  name: 'keyVaultResourcePreservingAccessPolicies_${uniqueString(keyVaultName)}'
  params: {
    location: location
    keyVaultName: keyVaultName
  }
}

File 2: KeyVaultResourcePreservingAccessPolicies.bicep:

param location string
param keyVaultName string

module resourceExistsModule 'ResourceExists.bicep' = {
  name: 'resourceExists_${uniqueString(keyVaultName)}'
  params: {
    location: location
    resourceName: keyVaultName
    // User assigned managed identity that is used to execute the deployment script on the resource group
    // User assigned Managed Identity info: https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview
    // User assigned Managed Identity needs reader role on the resource group of the resource (See Access Policies of the resource group)
    identityPrincipalId: 'MyUserAssignedManagedIdentity'
  }
}

module keyVaultModule 'KeyVaultResourceUsingExistingAccessPolicies.bicep' = {
  name: 'keyVaultResourceUsingExistingAccessPolicies_${uniqueString(keyVaultName)}'
  params: {
    location: location
    keyVaultName: keyVaultName
    keyVaultResourceExists: resourceExistsModule.outputs.exists
  }
}

File 3: KeyVaultResourceUsingExistingAccessPolicies.bicep

param location string

param keyVaultName string
param keyVaultResourceExists bool

resource existingKeyVaultResource 'Microsoft.KeyVault/vaults@2021-11-01-preview' existing = if (keyVaultResourceExists) {
  name: keyVaultName
}

module keyVaultModule 'KeyVaultResource.bicep' = {
  name: 'KeyVaultResource_${uniqueString(keyVaultName)}'
  params: {
    location: location
    keyVaultName: keyVaultName
    accessPolicies: keyVaultResourceExists ? existingKeyVaultResource.properties.accessPolicies : []
  }
  dependsOn: [
    existingKeyVaultResource
  ]
}

File 4: ResourceExists.bicep:

// Based on https://github.com/olafloogman/BicepModules/blob/main/resource-exists.bicep and https://ochzhen.com/blog/check-if-resource-exists-azure-bicep

@description('Resource name to check in current scope (resource group)')
param resourceName string

@description('Resource ID of user managed identity with reader permissions in current scope')
param identityPrincipalId string

param location string
param utcValue string = utcNow()

var userAssignedIdentity = resourceId(subscription().subscriptionId, resourceGroup().name, 'Microsoft.ManagedIdentity/userAssignedIdentities', '${identityPrincipalId}')

// The script below performs an 'az resource list' command to determine whether a resource exists
resource resource_exists_script 'Microsoft.Resources/deploymentScripts@2020-10-01' = {
  name: 'resourceExistsDeploymentScript_${resourceName}'
  location: location
  kind: 'AzureCLI'
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${userAssignedIdentity}': {}
    }
  }
  properties: {
    forceUpdateTag: utcValue
    azCliVersion: '2.15.0'
    timeout: 'PT10M'    
    scriptContent: 'result=$(az resource list --resource-group ${resourceGroup().name} --name ${resourceName}); echo $result; echo $result | jq -c \'{Result: map({name: .name})}\' > $AZ_SCRIPTS_OUTPUT_PATH;'
    cleanupPreference: 'OnSuccess'
    retentionInterval: 'P1D'
  }
}

//Script returns something like: //[{"name":"MyKeyVault"}]
output exists bool = length(resource_exists_script.properties.outputs.Result) > 0

File 5: KeyVaultResource.bicep

param location string
param keyVaultName string
param accessPolicies array

resource keyVaultResource 'Microsoft.KeyVault/vaults@2021-11-01-preview' = {
  name: keyVaultName
  location: location
  properties: {
    sku: {
      family: 'A'
      name: 'standard'
    }
    accessPolicies: accessPolicies
    tenantId: subscription().tenantId
    enableSoftDelete: true
    softDeleteRetentionInDays: 90
    enableRbacAuthorization: false
    enablePurgeProtection: true
    publicNetworkAccess: 'Enabled'
  }
}

Answer 2

The issue with the accepted answer is that it removes the key vault from the ARM template altogether, meaning that the key vault's creation becomes a manual process on new environments.

ARM does not allow a key vault to be redeployed without clearing its existing access policies. The accessPolicies property is required (except when recovering a deleted vault), so omitting it will cause an error. Setting it to [] will clear all existing policies. There has been a Microsoft Feedback request to fix this since 2018, currently with 152 votes.

The best way I've found of working around this is to make the key vault deployed conditionally only if it does not already exist, and define the access policies through a separate add child resource. This causes the specified policies to get added or updated, whilst preserving any other existing policies. I check whether the key vault already exists by passing in the list of existing resource names to the ARM template.

In the Azure pipeline:

- task: AzurePowerShell@5
  displayName: 'Get existing resource names'
  inputs:
    azureSubscription: '$(armServiceConnection)'
    azurePowerShellVersion: 'LatestVersion'
    ScriptType: 'InlineScript'
    Inline: |      
      $resourceNames = (Get-AzResource -ResourceGroupName $(resourceGroupName)).Name | ConvertTo-Json -Compress
      Write-Output "##vso[task.setvariable variable=existingResourceNames]$resourceNames"
    azurePowerShellVersion: 'LatestVersion'

- task: AzureResourceManagerTemplateDeployment@3
  name: DeployResourcesTemplate
  displayName: 'Deploy resources through ARM template
  inputs:
    deploymentScope: 'Resource Group'
    action: 'Create Or Update Resource Group'
    # ...
    overrideParameters: >-
      -existingResourceNames $(existingResourceNames)
      # ...
    deploymentMode: 'Incremental'

In the ARM template:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",

  "parameters": {
    "keyVaultName": {
      "type": "string"
    },
    "existingResourceNames": {
      "type": "array",
      "defaultValue": []
    }
  },

  "resources": [
    {
      "type": "Microsoft.KeyVault/vaults",
      "apiVersion": "2016-10-01",
      "name": "[parameters('keyVaultName')]",
      "location": "[resourceGroup().location]",
      // Only deploy the key vault if it does not already exist.
      // Conditional deployment doesn't cascade to child resources, which can be deployed even when their parent isn't.
      "condition": "[not(contains(parameters('existingResourceNames'), parameters('keyVaultName')))]",
      "properties": {
        "sku": {
          "family": "A",
          "name": "Standard"
        },
        "tenantId": "[subscription().tenantId]",
        "enabledForDeployment": false,
        "enabledForDiskEncryption": false,
        "enabledForTemplateDeployment": true,
        "enableSoftDelete": true,
        "accessPolicies": []
      },
      "resources": [
        {
          "type": "accessPolicies",
          "apiVersion": "2016-10-01",
          "name": "add",
          "location": "[resourceGroup().location]",
          "dependsOn": [
            "[parameters('keyVaultName')]"
          ],
          "properties": {
            "accessPolicies": [
              // Specify your access policies here.
              // List does not need to be exhaustive; other existing access policies are preserved.
            ]
          }
        }
      ]
    }
  ]
}

Answer 3

We found a work-around for this by using Managed identity

Example for Bicep:

• Create a User Assigned Identity Resource:

   resource UserAssignedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
    name: 'UserAssignedIdentityName'
    ...
   }
  

• And then add the object id to the key vault deployment and include the access:

   resource keyVault 'Microsoft.KeyVault/vaults@2020-04-01-preview' = {
    ...
    properties: {
      ...
      accessPolicies: [
        {
          ...
          objectId: UserAssignedIdentity.properties.principalId // Managed Identity
          permissions: {
            ...
          }
        }
      ]
    }
   }
  

• When deploying, for an example, a function app from another arm template that requires access to the Key Vault. Use an existing User Assigned Identity:

   resource UserAssignedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' existing = {
    name: 'Name of an existing UAID resource'
   }
  
   resource FunctionApp 'Microsoft.Web/sites@2020-12-01' = {
    ...
    identity:{
      type: 'SystemAssigned, UserAssigned'
      userAssignedIdentities:{
        '${UserAssignedIdentity.id}' :{}
      }
     ...
   }
  

Answer 4

We follow the below workaround solution to do idempotent key vault ARM deployment.

1.Deploying Resource group tags used as checkpoint. In Parameters block

"resourceTags": {
  "type": "object",
  "defaultValue": {
    "DeploymentLabel": "1"
  }
}

In Resource block

{
  "name": "default",
  "type": "Microsoft.Resources/tags",
  "apiVersion": "2020-10-01",
  "properties": {
    "tags": "[parameters('resourceTags')]"
  }
}

2. Now Conditional deployment can be used, if only one tag exist:

this means only Resource group is created and yet to create the Keyvault.

{
  "apiVersion": "2019-09-01",
  "name": "[parameters('keyVaultName')]",
  "location": "[resourceGroup().location]",
  "condition": "[equals(variables('tagslength'), 1)]",
  "type": "Microsoft.KeyVault/vaults",
 }

In the same keyvault arm template, add new tag to register the for Keyvault deployment. Now, there are 2 tags added to RG, so we could redeploy without losing access policies.

Answer 5

With the KeyVault's apiVersion 2019-09-01 you can workaround this issue by deploying the vault (type: Microsoft.KeyVault/vaults) only when it's new (using condition).

The access-policies can then be defined separatly as a sub-resource using type Microsoft.KeyVault/vaults/accesspolicies

Answer 6

This is the behavior you should get, as ARM templates are idempotent. you will be able to alter this behavior if you create access policies as a separate resource:

{
  "name": "vaultName/policyName",
  "location": xxx,
  "api-version": "2016-10-01",
  "type": "Microsoft.KeyVault/vaults/accessPolicies",
  "properties": {
    "accessPolicies": [
      {
        "tenantId": "00000000-0000-0000-0000-000000000000",
        "objectId": "00000000-0000-0000-0000-000000000000",
        "permissions": {
          "keys": [
            "encrypt"
          ],
          "secrets": [
            "get"
          ],
          "certificates": [
            "get"
          ]
        }
      }
    ]
  }
}

keep in mind this is a rough sketch, it might not work, but you can get it to work fairly easily. it is to illustrate the idea.

reference: https://learn.microsoft.com/en-us/rest/api/keyvault/vaults/updateaccesspolicy