CODEWITHSUNDEEP
rcurldownload
user3423306
user3423306

Reputation: 31

Warning message In download.file: download had nonzero exit status

I am downloading data from data.gov website and I get following two types of errors in the process:

fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families"
download.file(fileUrl,destfile=".data/studentdata.csv",method="curl")
Warning message:
In download.file(fileUrl, destfile = ".data/studentdata.csv", method = "curl") :
  download had nonzero exit status

I tried to remove the method="curl" as suggested in other forum, but again I get this new error

download.file(fileUrl,destfile=".data/studentdata.csv")
Error in download.file(fileUrl, destfile = ".data/studentdata.csv") : 
  cannot open destfile '.data/studentdata.csv', reason 'No such file or directory'

Upvotes: 3

Views: 6729

Answers (5)

MrBoneClinkz
MrBoneClinkz

Reputation: 41

I think there are two major factors why your curl doesn't work well.

First, the problem is on your URL. fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families". In your URL, it is not referred to a csv file. So, they won't work even if you set the destination into a csv file such as destfile = ".data/studentdata.csv"

I have an example of getting a csv dataset using the same code (different dataset):

  1. DataURL<- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD" (This link refers to a rows.csv file)
  2. download.file(DataURL, destfile="./data/rows.csv", method="curl") (The method is quite same, using curl)

Second, previously I had the same problem that the curl does not work, even I used a proper URL that refers to a csv file. However, when I diagnosed a bit deeper, I found something interesting fact about why my curl method cannot work properly. It was my R session program. I used a 32-bit R, in which the error occurs. Later then, I tried to change the session into a 64-bit R. Amazingly, and the download status was running at that time. To see your R session architecture (whether you are using 32-bit or 64-bit), type in your R:

sessionInfo()

R version 3.5.3 (2019-03-11)

Platform: x86_64-w64-ming32/x64 (64-bit)

Running under: Windows >= 8 x64 (build 9200)

You have to switch your R, from 32-bit to 64-bit to avoid 'curl' call had nonzero exit status. You go to your R directory folder, and then you run a 64-bit R.

If you are using a Windows OS and installing the R in a default path folder, you can run this C:\Program Files\R\R-3.5.3\bin\x64\R.exe. (I used a version of 3.5.3, so it may be different with your version)

If you are using R-studio, you can switch the R session on the menubar Tools -> Global Options -> R version -> Change -> Use your machine's default version of R64 (64-bit) -> OK. Then restart your R-studio.

However, it depends on your OS architecture. If you are using a 32-bit OS, hence you have to find another way to solve this.

Upvotes: 1

Evans Rono
Evans Rono

Reputation: 21

Try this: file<-'http://catalog.data.gov/dataset/expenditures-on-children-by-families'

file<- read.csv(file)

Upvotes: 0

Sheriseology
Sheriseology

Reputation: 334

I was having the same problem.

Then I realized that I forget to create the "data" directory!

So try adding this above your fileURL line to create the directory first.

if(!file.exists("data")){
  dir.create("data")
}

Also, if you are running a Mac, then you want to keep method="curl" when downloading a https file. I don't believe Windows has that problem hence the suggestions to remove it.

Upvotes: 0

Janak Mayer
Janak Mayer

Reputation: 116

If I'm not very much mistaken you just have a simple typo here. I suspect you have a "data" directory, not a ".data" directory - in which case your only problem is that your destfile string needs to begin "./data", not ".data".

Upvotes: 0

jlhoward
jlhoward

Reputation: 59425

So looking at the code for download.file(...), if you specify method="curl" the function tries to use the curl shell command. If this command does not exist on your system, you will get the error above.

If you do not specify a method, the default is to use an internal R method to download, which evidently works on your system. In that case, the function is trying to put the file in .data/studentdata.csv but evidently there is not .data directory. Try taking out the ..

When this download works, you will get a text/html file, not a csv file. Your url points to a web page, not a download link. That page does have a download link, but unfortunately it is a pdf, not a csv.

Finally, if your goal is to have the data in R (is it?), and if the link actually produces a csv file, you could more easily use

df <- read.csv(fileUrl)

Upvotes: 0

Related Questions