Terraform Databricks Storage credential for an Access Connector with User Assigned managed identity

Question

I have a databricks access connector, that I created for accessing external locations . The access connector created has a user defined managed identity (not a system assigned one).

Now when I try to create a storage credential in Terraform using that access connector, I get an error saying

cannot create storage credential: Azure Managed Identity Credential with Access Connector Id nameofcred  could not be found

Using this:

resource "databricks_storage_credential" "storage_credential" {
  name = "example_cred"
  azure_managed_identity {
    access_connector_id = <entered the resource id for the Access Connector for Azure Databricks>
  }
  comment = "Managed identity credential managed by TF"
}

And this is how my access connector looks like (configured with userAssigned Identity)

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "accessConnectors_ac_connector_rxample": {
            "defaultValue": "example_name",
            "type": "String"
        }
    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.Databricks/accessConnectors",
            "apiVersion": "2023-05-01",
            "name": "[parameters('accessConnectors_ac_connector_example_name')]",
            "location": "northeurope",
            "identity": {
                "type": "UserAssigned",
                "userAssignedIdentities": {
                    "/subscriptions/xxxxxxxxxxxxx/resourceGroups/rg-example/providers/Microsoft.ManagedIdentity/userAssignedIdentities/userassignedminame": {}
                }
            },
            "properties": {}
        }
    ]
}

This is the terraform resource link (Azure is the cloud provider): here

Is User Assigned managed identity not supported?

I also tried with azure_service_principal block with directory_id and application_id, but it keeps failing because client_secret is a required property, and since this is a user defined managed identity, I can't create secrets (it is just listed as an SPN aka enterprise application in Azure AD).

Is this not supported?

From the GUI it is supported:

Answer 1

You should upgrade databrick provider to the lastest version and managed_identity_id into azure_managed_identity block.

Like this:

resource "databricks_storage_credential" "storage_credential" {
  name = "example_cred"
  azure_managed_identity {
    access_connector_id = <entered the resource id for the Access Connector for Azure Databricks>
    managed_identity_id = <resource id of Managed Identity>
  }
  comment = "Managed identity credential managed by TF"
}

Doc: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/storage_credential

Answer 2

I also tried with azure_service_principal block with directory_id and application_id, but it keeps failing because client_secret is a required property, and since this is a user defined managedidentity, I can't create secrets (it is just listed as an SPN aka enterprise application in Azure AD).

User Assigned Managed Identities are not supported in Terraform for creating storage credentials.

Storage credential represents an authentication and authorization mechanism for accessing data stored on your cloud tenant, using either an Azure managed identity or a service principal. follow the Ms Doc about Storage credential.

Alternatively, you can create storage credentials using a service principal by following the steps below.

Create a service principal in the Azure portal and provide it access to your storage account.

1. Create a client secret for the service principal and note down the directory ID , application ID and Client Secret for the service principal.
2. Log into your storage account and assign the service principal the Storage Blob Data Contributor role. Follow the instructions in the MS doc about Data bricks storage credentials using service prinicipal

I created a Databricks storage credential using a service principal using Terraform code

   provider "azurerm" {
  features {}
}

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
    }
    databricks = {
      source = "databricks/databricks"
    }
  }
}

# Use Databricks CLI authentication.
provider "databricks" {
profile = "DEFAULT"
host    = "Databricks-URL"
}
resource "databricks_storage_credential" "storage_credential" {
  name = "databricks-storage"
  azure_service_principal {
    application_id   = ""
    directory_id     = ""
    client_secret   = ""
  }
  comment = "Service Principal credential managed by TF"
}

Terraform Plan:

Terraform Apply: