CODEWITHSUNDEEP
pythonazure-machine-learning-service
Joanteixi
Joanteixi

Reputation: 537

Upload dataframe as dataset in Azure Machine Learning

after download a dataset, convert to dataframe and manipulate it.. how can I upload again as new dataset in Azure Machine Learning?

Upvotes: 2

Views: 5912

Answers (1)

May Hu
May Hu

Reputation: 501

You can follow the steps below:
1. write dataframe to a local file (e.g. csv, parquet)

local_path = 'data/prepared.csv'
df.to_csv(local_path)
  1. upload the local file to a datastore on the cloud
# azureml-core of version 1.0.72 or higher is required
# azureml-dataprep[pandas] of version 1.1.34 or higher is required
from azureml.core import Workspace, Dataset

subscription_id = 'xxxxxxxxxxxxxxxxxxxxx'
resource_group = 'xxxxxx'
workspace_name = 'xxxxxxxxxxxxxxxx'

workspace = Workspace(subscription_id, resource_group, workspace_name)
# get the datastore to upload prepared data
datastore = workspace.get_default_datastore()
# upload the local file from src_dir to the target_path in datastore
datastore.upload(src_dir='data', target_path='data')
  1. create a dataset referencing the cloud location
ds = Dataset.Tabular.from_delimited_files(datastore.path('data/prepared.csv'))

Upvotes: 7

Related Questions