CODEWITHSUNDEEP
azureazure-devopsazure-pipelinesazure-rm-template
abautista
abautista

Reputation: 2780

How to create a folder inside of an Azure Data Lake container using an ARM template?

I have an Azure ADLS storage account called eventcoadltest and I have a container called eventconnector-transformed-data-fs.

enter image description here

I have deployed this ADLS through an ARM template but I need to create a directory inside of eventconnector-transformed-data-fs as shown below (the folder debugging was created through the UI but I need to achieve the same with an ARM template):

enter image description here

I have found some posts that indicate this is not possible but it can be bypassed with some workarounds:

  1. How to create empty folder in azure blob storage
  2. Use ARM template to create directories in Azure Storage Containers?
  3. How to create a folder inside container in Azure Data Lake Storage Gen2 with the help of 'azure-storage' Package
  4. ARM template throws incorrect segments lengths for array of storage containers types
  5. how to create blob container inside Storage Account using ARM templates
  6. Microsoft Azure: How to create sub directory in a blob container
  7. How to create an azure blob storage using Arm template?
  8. How to create directories in Azure storage container without creating extra files?
  9. How to create a folder inside container in Azure Data Lake Storage Gen2 with the help of 'azure-storage' Package

I have tried to modify my ARM template as well to achieve a similar result but I haven't had any success.

{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
    "storageAccountDLName": {
        "type": "string"
    },
    "sku": {
        "type": "string"
    },
    "directoryOutput":{
        "type": "string"
    }
},
"resources": [
    {
        "type": "Microsoft.Storage/storageAccounts",
        "apiVersion": "2021-02-01",
        "sku": {
            "name": "[parameters('sku')]",
            "tier": "Standard"
        },
        "kind": "StorageV2",
        "name": "[parameters('storageAccountDLName')]",
        "location": "[resourceGroup().location]",
        "tags": {
            "Contact": "[parameters('contact')]"
        },
        "scale": null,
        "properties": {
            "isHnsEnabled": true,
            "networkAcls": {
                "bypass": "AzureServices",
                "virtualNetworkRules": [],
                "ipRules": [],
                "defaultAction": "Allow"
            }
        },
        "dependsOn": [],
        "resources": [
            {
                "type": "storageAccounts/blobServices/containers",
                "name": "[concat('default/', 'eventconnector-raw-data-fs/test')]",
                "apiVersion": "2021-02-01",
                "properties": {},
                "dependsOn": [
                    "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountDLName'))]"
                ]
            }
        ]
    }
]
}

The following code was modified for trying to create the folders inside of the containers.

"type": "storageAccounts/blobServices/containers",
"name": "[concat('default/', 'eventconnector-raw-data-fs/test')]"

The reason why I am trying to solve this problem is because I won't have access to create folders in our production environment, so that's why I need to do the deployment fully through ARM. How can I create this folder with the deployment script? Is there another alternative for achieving my desired result? Any idea or suggestion is welcome :)

Upvotes: 0

Views: 2525

Answers (2)

abautista
abautista

Reputation: 2780

After some research I found out that it is possible to create a folder via Databricks with the following command:

dbutils.fs.mkdirs("dbfs:/mnt/folder_desktop/test/uploads")

I had to configure Databricks with my Azure Datafactory in order to run this command.

Upvotes: 0

4c74356b41
4c74356b41

Reputation: 72181

this doesn't make any sense, as you can not create folders in Azure Storage. They don't have folders. blobs are individual objects\entities. you are confused to believe folders exist, because UI renders them as folders, however THERE ARE NO FOLDERS in a Azure Storage Blob Container.

TLDR: you can not do this at all no matter how hard you try

Upvotes: 0

Related Questions