CODEWITHSUNDEEP
pythonazureazure-storageazure-blob-storageazure-eventhub
Alexander Farber
Alexander Farber

Reputation: 23018

How to list the newest AVRO blobs in a storage account of Event Hub?

We have an Event Hub instance in Microsoft Azure which captures data in AVRO format into a blob storage account:

event hub

The blob names are container1/my-test-namespace/my-test-eventhub followed by 0, 1, 2 or 3 - and then the date.

I am able to list the blobs using the ContainerClient python class:

import os, uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, __version__

print("Azure Blob storage v" + __version__ + " - List blobs in my-test container1")
connect_str = 'DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net'

container_client = ContainerClient.from_connection_string(connect_str, 'container1')
blob_list = container_client.list_blobs()
i = 0
for blob in blob_list:
    print(str(i) + ":\t" + blob.name)
    i = i + 1
    if i >= 10:
        break

This works well -

cmd

However I wonder, how to display the 10 newest blobs?

I would like to list the 10 most recent blobs, having ".avro" at the end.

I have searched in the azure.storage.blob documentation, but haven't found a way yet.

If you look at the top screenshot, there is a "Modified" column, I wonder if it would be useable from the python script.

Upvotes: 0

Views: 135

Answers (1)

Ivan Glasenberg
Ivan Glasenberg

Reputation: 30015

Unfortunately, there is no sort method in azure blob storage sdk.

There're 2 workarounds.

The first one is the you can create an blob triggered azure function, and when a new blob is created in that container, write the related information(like blob_name, creation_time) into a database etc. Then you can query the latest 10 blobs by accessing the database. This is recommended as of now.

The 2nd one is that, you should list all the blobs, then write your own code to sort them by creation time property.

Upvotes: 1

Related Questions