CODEWITHSUNDEEP
rerror-handlingtry-catch
Dail
Dail

Reputation: 4606

All the connections are in use: Execution halted

I'm using getYahooData() function in TTR package quite intensely.

I have this piece of code:

for(i in 1:nrow(symbol)){
    tryCatch(prices <- getYahooData(symbol$symbol[i], from, to, freq="daily", 
                                    type="price"), 
             warning=function(e) continue <- 0)
    if (continue==0) next
}

This loop is long I get this error:

Error in file(file, "rt") : all connections are in use Calls: tryCatch ... doTryCatch -> getYahooData -> getYahooData -> read.table -> file Execution halted

What can I do?

UPDATE:

If i use closeAllConnections() I get:

 I get: *** caught segfault *** address (nil), cause 'memory not mapped' Traceback: 1: getConnection(set[i]) 2: close(getConnection(set[i])) 3: closeAllConnections() aborting ...

Upvotes: 11

Views: 13181

Answers (4)

Robert Casey
Robert Casey

Reputation: 1581

Somewhere in my past digging, someone mentioned that the use of a URL reference in a function like read.table.url() or url() invoked a somewhat buggy native R HTTP connector. What has proven to work better for closure of connections on error is to explicitly call RCurl inside of a read.table(), or equivalent, function call. For instance, this was giving me problems when a lot of HTTP errors accumulated:

result <- try ( DF <- read.table(con <- url(url), col.names=colNames),
silent=TRUE )

I have seen good results going with the RCurl variation, calling its getURL function:

result <- try ( DF <- read.table(textConnection(getURL(url)), col.names=colNames),
silent=TRUE )

This is while running R v2.15.3.

Upvotes: 0

Matt Shotwell
Matt Shotwell

Reputation: 341

This is indeed a bug in the R source code regarding how connections are registered. I've posted some comments and a patch at the R Bugzilla site here: http://bugs.r-project.org/bugzilla3/show_bug.cgi?id=14660. Joris' recommendations are sound. However, closeAllConnections() will also work, when the bug is fixed. (P.S. this is my first StackOverflow post. Pardon me if I've violated etiquette.)

Upvotes: 9

Joris Meys
Joris Meys

Reputation: 108533

First : do not ever in your life use that continue construct again. It's of no use. tryCatch() will continue if you defined a handler for an error or a warning. It will use that one instead of the "default" error=function(e) stop(e). This one will stop your function. If you define a handler (either warning= or error= ), your script will not be stopped, so that continue is not necessary.

This said : The correct use of tryCatch in this case would be :

for(i in 1:nrow(symbol)){

tryCatch(prices <- getYahooData(symbol$symbol[i], from, to, freq="daily",
                                    type="price"), error = function(e){})

}

or, if you use it in a script and want to go to the next loop when an error occurs, you can simply use :

for(i in 1:nrow(symbol)){

    prices <- try(getYahooData(symbol$symbol[i], from, to, freq="daily",
                                    type="price"), silent=TRUE)

    if(inherits(prices,"try-error")) { next } # only true if an error occurs
    ... # rest of calculations
}

If you would have used this way of tryCatch or try, you wouldn't have had the problems you report here.

Now I could reproduce your case, if I use nonexistent symbols. Your erroneous use of tryCatch() function is causing you trouble. read.table returns an error (Error in file(file, "rt") : cannot open the connection). This is an error, not a warning. You get an additional warning saying that a 404 File not found was returned.

When a warning is issued together with an error, the handler function for the warning is dealt with first. That's because a warning has to be thrown before the function can be stopped. So it won't handle the error you get, which means that the on.exit(close(file)) in read.table() will not to be called. Hence, the connection is not closed correctly and still considered open, although it can't be found anymore by R (showAllConnections() shows nothing). As the error is not dealt with, something goes wrong in the registration of the connections. As the connection couldn't be opened, the on.exit(close(...)) will have no effect. showConnections() doesn't show the connection, but somehow R still thinks it's there. Hence, all hell breaks loose and you crash your R.

thanks for the corrections to @Tommy

A simple code example to illustrate this :

myfun <- function(x){
   if(x>1) warning("aWarning")
   stop("aStop")
   x
}

tryCatch(myfun(0.5),
          warning=function(w)print("warning"),
          error=function(e) print("stop"))
[1] "stop"              

tryCatch(myfun(1.5),
          warning=function(w)print("warning"),
          error=function(e) print("stop"))
[1] "warning"

In summary :

  • check the symbols you use. They're probably wrong.
  • do not ever again use a warning handler if you expect errors.

And as an extra: your loop will only return the result of the last call, as you overwrite prices every time you go through the loop, in case you would have used correct symbols.

Edit : in case you want to continue the action

Upvotes: 16

IRTFM
IRTFM

Reputation: 263342

Close some connections? Could be as easy as inserting closeAllConnections() at the end of that loop body.

Upvotes: 14

Related Questions