CODEWITHSUNDEEP
rapiloops
SimonCW
SimonCW

Reputation: 356

R: Looping through list and passing values to API of UN Comtrade. + Programming a pause in the loop

I want to pass the arguments of a list to a function calling the API of the UN Comtrade Data. Also, I need to add pauses after each query to the API to not exceed the restrictions of UN Comtrade.

Without the loop I call the API this way:

q276a <- get.Comtrade(r="276", p="0", freq="M", ps="all", cc="842520", rg="2", fmt="csv") df_q276a<-as.data.frame(do.call(rbind, q1a).

r="276" is specifying that I want the data from the Reporting Country with the code "276" which is Germany. To get data from all reporting countries I want to loop through a list with all the country codes. (I can't just specify "all" since the API restricts the value "all" for only one of the paramters r, p, ps)

What I've written so far is: 

for (i in ls_reporters){
  assign(paste("q", i,"a", sep = ""), get.Comtrade(r="i", p="0", freq="M", ps="all", cc="842520", rg="2", fmt="csv"))
}

This produces the error

Error in file(file, "rt") : cannot open the connection

My code seems to pass r="i" to the API instead of cycling through the values in ls_reporters und passing these to the API, e.g. 4 then 18 and so on. However, if I type i I get "4" which is the first value of the ls_reporters.

As additional Question: How can i program a pause of a few seconds after each iteration to not exceed the 1 request per second restriction of UN Comtrade?

If there is a more elegant way programming this without loops I would be glad to learn it. I am pretty new to R.

The R code to query data through the public API can be found here. The documentation for the API can be found here

Upvotes: 1

Views: 1601

Answers (1)

Parfait
Parfait

Reputation: 107687

Simply use lapply where you pass i as on object reference and not a literal with quotes (if ls_reporters holds numeric data, wrap i in as.character()). Also, use use Sys.sleep(time) to pause execution of script between iterations, time measured in seconds.

Do note, the returned output unlike a for loop is a large list of data elements equal to the number of elements in ls_reporters. Should you need individual objects for each country data, use list2env() (but no need to flood environment with potentially 100s of objects if not needed):

# LIST OF COMTRADE DATA (LENGTH EQUAL TO INPUT LIST)
comtrade_data <- lapply(ls_reporters, function(i) {
                            Sys.sleep(1)
                            tmp <- get.Comtrade(r=i, p="0", freq="M", ps="all", 
                                                cc="842520", rg="2", fmt="csv")
                            df <- as.data.frame(do.call(rbind, tmp))
                            return(df)
                 })

# NAME EACH ELEMNT OF NEW LIST BY PREFIXING "q"
comtrade_data <- setNames(comtrade_data, paste0("df_q", ls_reporters))

# CREATE df_q1, df_q2, df_q3, ... AS INDIVIDUAL OBJECTS
list2env(comtrade_data, envir=.GlobalEnv)

# EVEN ROW BIND TO DATAFRAME (IF ELEMENTS HAVE SAME EXACT ONE-LEVEL DATA STRUCTURE)
df <- do.call(rbind, comtrade_data)

Upvotes: 1

Related Questions