CODEWITHSUNDEEP
rurl
zhiwei li
zhiwei li

Reputation: 1711

How to batched download data from URL in R

I have many of urls. Each url points to a csv file and each csv file has its own name.

I want to download data from the urls and save it on my computer.

I have tried the code in batch download zipped files in R, but failed.

So I want to know if there is an easy way to batched download data from URL and save them on computer.

urls = c(
'http://minio.tapdata.org.cn:9000/tap-bj-1km/input_v3/Tile_162_lonlat.csv.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=24BJXNVDJVVCUTC9CQZ1%2F20220418%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220418T011215Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=3d203f787748209654fc863992c6b51f206df3146dd8054cf8b4aea1ffc9150f',
'http://minio.tapdata.org.cn:9000/tap-bj-1km/input_v3/2010/1/China_PM25_1km_2010_001_162.csv.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=24BJXNVDJVVCUTC9CQZ1%2F20220418%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220418T043413Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=f3f8e4bbac9227e30e314dc8dd4dc0802a3e54719a07a7754ccae4609f0df330',
'http://minio.tapdata.org.cn:9000/tap-bj-1km/input_v3/2010/1/China_PM25_1km_2010_002_162.csv.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=24BJXNVDJVVCUTC9CQZ1%2F20220418%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220418T043413Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=c27655cfbb61a7a6f9c9cb2c2e01624037e84c5bc0c4aecb59bb2975e2c21466',
'http://minio.tapdata.org.cn:9000/tap-bj-1km/input_v3/2010/1/China_PM25_1km_2010_003_162.csv.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=24BJXNVDJVVCUTC9CQZ1%2F20220418%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220418T043413Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=c7baba0660d28263033dc0df5db4cdb851d1a7b6a36e5b369e3dfe658b8f5305'
)
df_urls = data.frame(url = urls) # all the url save in r data frame

Upvotes: 0

Views: 349

Answers (1)

Nad Pat
Nad Pat

Reputation: 3173

To download files in your working directory we can use downloader package. Extract zip file name using gsub from urls.

library(downloader)
lapply(urls, function(x){
#create zip file name
  nam = gsub(".*[/]([^.]+)[.].*", "\\1", x)
  nam = paste0(nam, '.zip')
#download zip files to your working directory. 
  download.file(x, nam, mode = 'wb')
})

Upvotes: 1

Related Questions