CODEWITHSUNDEEP
azureazure-storage-files
arao6
arao6

Reputation: 3376

Azure Files - identify file uploads in progress?

While listing files and directories, I also get files that are still in the progress of uploading. The file size shows up as the full size, which I assume is because Azure reserves space. Is there a way to only list files that have been fully uploaded in the storage account? Or perhaps any other way to implement locking? I have no control over the uploader side of this, so I can't implement locking from an application perspective.

Upvotes: 0

Views: 60

Answers (1)

Venkatesan
Venkatesan

Reputation: 10455

Azure Files - identify file uploads in progress?

Azure file does not have a built-in mechanism specifically designed to detect files in the process of uploading.

Alternatively, you can use Azure Python SDK to list files in an Azure File Share and check for the LastModified timestamp to infer whether a file has been fully uploaded.

Code:

from azure.storage.fileshare import ShareServiceClient
from datetime import datetime, timedelta,timezone

# Replace these with your actual values
connection_string = "xxxx"
share_name = "share1"
directory_name = "sample"

# Create the service client
service_client = ShareServiceClient.from_connection_string(connection_string)
share_client = service_client.get_share_client(share_name)
directory_client=share_client.get_directory_client(directory_name)

threshold = timedelta(minutes=5)

files = directory_client.list_directories_and_files()

# Check each file's LastModified timestamp
for file in files:
    if not file.is_directory:  # We are only interested in files, not directories
        file_client = directory_client.get_file_client(file.name)
        
        # Get file properties
        properties = file_client.get_file_properties()
        last_modified = properties['last_modified']
        current_time = datetime.now(timezone.utc)
        # Compare last_modified timestamp with the current time
        if last_modified and (current_time - last_modified) > threshold:
            print(f"File '{file.name}' has been fully uploaded. Last modified: {last_modified}")
        else:
            print(f"File '{file.name}' is still uploading or modified recently. Last modified: {last_modified}")

The above code checks files in a specified directory for their last modification time. It compares the LastModified timestamp of each file with the current time to determine if the file has been uploaded or is still being uploaded. If the file hasn't been modified in the last 5 minutes, it considers the file fully uploaded and prints the result.

Output:

File 'random_records.csv' is still uploading or modified recently. Last modified: 2024-11-25 07:43:25+00:00  
File 'Sleep-Away.wav' has been fully uploaded. Last modified: 2024-11-25 07:34:02+00:00
File 'vbaupload.gif' has been fully uploaded. Last modified: 2024-11-25 07:34:05+00:00

enter image description here

Upvotes: 1

Related Questions