CODEWITHSUNDEEP
pythonpython-3.xazureazure-storageazure-blob-storage
quervernetzt
quervernetzt

Reputation: 11651

Azure Storage: Blob: Python: Get indicator if there are blobs at all

I have a Python app. In this context I want to retrieve the blob references from an Azure Storage container that match a certain prefix and then delete all the blobs in one go. I tried the following:

container_client: ContainerClient = ContainerClient.from_connection_string(conn_str=storage_account_connection_string, container_name=container_name)

blob_list: ItemPaged[BlobProperties] = container_client.list_blobs(name_starts_with=prefix)

container_client.delete_blobs(*blob_list, delete_snapshots="include")

This works fine as long as there are blobs that match the prefix. But if that is not the case I get an exception when trying to execute delete_blobs:

tuple index out of range

I don't want to work with try except and I also don't want to iterate first. I would like to have an indicator that tells me if there are blobs at all without the need to do extra calls.

How can I do that?

Thanks

EDIT: Based on what has been suggested by @Gaurav the following approach works:

from azure.storage.blob import ContainerClient, BlobProperties
from azure.core.paging import ItemPaged
from typing import List

blob_paged: ItemPaged[BlobProperties] = container_client.list_blobs(name_starts_with=prefix)
blob_list: List[dict] = list(blob_paged)
number_of_blobs: int = len(blob_list)

if number_of_blobs > 0:
    container_client.delete_blobs(*blob_list, delete_snapshots="include")
    log.debug(f"Deleted '{ number_of_blobs }' blobs and snapshots...")   
else:
    log.debug(f"No blobs to be deleted...")

Three things you should be aware about:

Upvotes: 3

Views: 3122

Answers (1)

Gaurav Mantri
Gaurav Mantri

Reputation: 136296

delete_blobs method makes use of Blob Batch operation to delete multiple blobs in a single request. According to the documentation, maximum number of items in a batch can be 256 or the maximum payload size is 4MB (Ref: https://learn.microsoft.com/en-us/rest/api/storageservices/blob-batch#remarks).

I believe you're getting this error is because you're either sending more than 256 blobs in your delete_blobs method or the payload is more than 4MB in size.

UPDATE

You will also get the error if the items in the blobs_list are zero. You can use the following code to see the number of items (Ref: Getting number of elements in an iterator in Python):

number_of_blobs = len(list(blobs_list))

Upvotes: 4

Related Questions