CODEWITHSUNDEEP
azureazure-blob-storageazure-data-factory
Stephen
Stephen

Reputation: 55

Azure Data Factory: Delete an Empty Directory Issue

Within an Azure Data Factory Pipeline I am attempting to remove an empty directory.

The files within the directory were removed by a previous pipelines' iterative operation thus leaving an empty directory to be removed.

The directory is a sub-folder: The hierarchy being:

container / top-level-folder (always present) / directory - dynamically created - the result of an unzip operation.

I have defined a specific dataset that points to

container / @concat('top-level-folder/',dataset().dataset_folder)

where 'dataset_folder' is the only parameter.

The Delete Activity is configured like this:

enter image description here

On running the pipeline it errors with this error:

Failed to execute delete activity with data source 'AzureBlobStorage' and error 'The required Blob is missing. Folder path: container/top level directory/Directory to be removed/.'. For details, please reference log file here:

The log is an empty spreadsheet.

What am I missing from either the dataset or delete activity?

Upvotes: 0

Views: 1136

Answers (2)

Stephen
Stephen

Reputation: 55

I used an Azure function suggested here: Delete folder using Microsoft.WindowsAzure.Storage.Blob : blockBlob.DeleteAsync();

to perform the action.

Upvotes: 0

Saideep Arikontham
Saideep Arikontham

Reputation: 6114

In azure blob storage, when all the contents inside a folder are deleted, then the folder would automatically get deleted.

  • When I deleted each file and tried to delete the folder in the end, then I got the same error.

enter image description here

  • This is because the folder is deleted as soon as the files in the folders are deleted. Only when using azure data lake storage, you will have to specifically delete the folder as well.

  • Since the requirement is to delete the folder in azure blob storage, you can simply remove the delete activity that you are using to remove the folder.

Upvotes: 0

Related Questions