CODEWITHSUNDEEP
pythonazureazure-storageazure-blob-storageazure-machine-learning-service
simong
simong

Reputation: 41

How to download folder from AzureML notebook folder to local or blob storage?

I saved file to the same directory using (./folder_name) when I use AzureML jupyter. Now how can I download to my local machine or blob storage?

The folder have a lot of files and sub-directory in it, which I scraped online. So it is not realistic to save one by one.

file_path = "./"

for i in target_key_word:
    tem_str = i.replace(' ', '+')
    dir_name = file_path + i
    if not os.path.exists(dir_name):
        os.mkdir(dir_name)
    else:    
        print("Directory " , dir_name ,  " already exists")

Upvotes: 4

Views: 12328

Answers (5)

wwnde
wwnde

Reputation: 26676

from azureml.core import Workspace, Datastore, Dataset

import os from datetime import datetime

Connect to your AzureML workspace

workspace_name = "<your_workspace_name>" resource_group = "<your_resource_group>" subscription_id = "<your_subscription_id>"

ws = Workspace.get( name=workspace_name, resource_group=resource_group, subscription_id=subscription_id )

Access the default datastore (or specify a custom one)

datastore = ws.get_default_datastore()

Path to the user's folder within the workspace

user_folder = "path/to/user/folder"

List files in the specified folder

all_files = datastore.blob_service.list_blobs(datastore.container_name, prefix=user_folder)

Extract file information (name and last modified)

file_details = [ { "name": blob.name, "last_modified": blob.properties.last_modified } for blob in all_files ]

Sort files by last modified date in descending order

sorted_files = sorted(file_details, key=lambda x: x["last_modified"], reverse=True)

Get the 5 latest modified files

latest_files = sorted_files[:5]

Print the details of the latest files

print("Latest 5 Modified Files:") for file in latest_files: print(f"File: {file['name']}, Last Modified: {file['last_modified']}")

Upvotes: 0

James
James

Reputation: 141

Appreciate this is an old question but for anyone else with this questions this is how I did it.

  1. Open a new terminal window in your Azure ML Notebooks homepage.

  2. Zip up the entire directory with:

     zip -r myfiles.zip .

  3. Refresh the folder explorer and download the zip file.

Upvotes: 13

Nina Sonneborn
Nina Sonneborn

Reputation: 52

Another option for downloading Notebooks is to go through the azure ML studio UI to "Edit in VS Code." Once you have it open in VS Code, right click the folder and there is an option "Download" You can save these files to local and then re-upload as needed.

Upvotes: 0

SoakingHummer
SoakingHummer

Reputation: 572

If you are asking how you could download all your files and folders from the Notebooks GUI of Azure ML Studio, I wish there was a straightforward way to do so. As far as I know, you could do the following for now:

  1. You have to download Azure Storage Explorer and navigate to the blob storage linked to your ML workspace in the Azure portal (you can find the precise blob storage account by navigating to the overview section of your ML workspace).
  2. After you open the linked blob storage from the resource group, navigate to the overview section where you should select Open in Explorer.
  3. Once the Storage Explorer is open you can link the Azure blob storage by connecting it to the Explorer (which is pretty straightforward, you just have to fetch the key from the Access Keys section).
  4. After your storage account is linked, navigate to the File Shares folder and find the folder prefixed code-. This is where you will find all your files and folders, which you can download to your local in bulk.

Upvotes: 3

Satya V
Satya V

Reputation: 4164

From my understanding you would like to download the project files :

You could do one of the below :

From the project page, click the "Download Project" button to download all the files of a given project.

OR

  1. Go to Azure Notebooks and sign in.

  2. From your public profile page, select My Projects at the top of the page.

  3. My Projects link on the top of the browser window

  4. Select the project.

  5. Click the download Download button to trigger a zip file download that contains all of your project files.

Upvotes: 0

Related Questions