CODEWITHSUNDEEP
razureparallel-processingrstudioazure-batch
Eric
Eric

Reputation: 111

How to make Azure batch see my data files?

I am new to Azure batch. I am trying to use R in parallel with Azure batch in rstudio to run code on a cluster. I am able to successfully start the cluster and get the example code to work properly. When I try to run my own code I am getting an error that says the cluster nodes cannot find my data files. Do I have to change my working directory to Azure batch somehow?

Any information on how to do this is much appreciated.

Upvotes: 0

Views: 77

Answers (1)

Eric
Eric

Reputation: 111

I have figured out how to get Azure batch to see my data files. Not sure if this is the most efficient way, but here is what I did.

  1. Download a program called Microsoft Azure Storage Explorer which runs on my local computer.
  2. Connect to my Azure storage using the storage name and primary storage key found in the Azure portal.
  3. in Microsoft Azure Storage Explorer find Blob containers, right click create new container.
  4. Upload data files to that new container.
  5. Right click on data files and go to copy URL.
  6. Paste URL in R like this model_Data<-read.csv(paste('https://<STORAGE NAME HERE>.blob.core.windows.net/$root/k',k,'%20data%20file.csv',sep=''),header=T)

Upvotes: 1

Related Questions